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MANAGEMENT  LEVERS  THAT  DRIVE  SERVICES  CONTRACTING 

SUCCESS 

ABSTRACT 

Contracting  for  services  in  the  Department  of  Defense  (DoD)  has  grown  over  the 
last  21  years.  This  growth  in  dollars  spent  has  brought  increased  political 
attention  and  scrutiny.  DoD  has  responded  to  problems  such  as  contract 
mismanagement  and  ill-defined  requirements  by  improving  service  acquisitions, 
but  it  still  has  problems.  The  problems  could  be  from  a  lack  of  standard  definition 
for  success.  Since  contract  success  and  failure  is  recorded  through  the  Contract 
Past  Performance  Assessment  Reporting  System  (CPARS)  this  information  is 
used  for  the  proxy  definition  for  success.  This  definition  was  used  to  address  the 
following  questions:  (1)  Do  the  types  of  services  being  acquired  affect  the 
success  of  a  service  contract,  (2)  Do  the  contractual  amounts  affect  the  success 
of  a  service  contract,  (3)  Does  the  level  of  competition  used  affect  the  success  of 
a  service  contract,  (4)  Does  the  contract  type  affect  the  success  of  a  services 
contract.  This  report  examined  715  CPARS  entries.  The  findings  revealed  that 
contractual  amounts  and  level  of  competition  affect  the  success  of  a  service 
contract.  The  findings  also  revealed  that  the  failure  rate  in  CPARS  is  lower  than 
expected.  From  these  findings,  the  report  presents  a  discussion  of  the  results 
and  managerial  implications,  and  recommends  an  alternate  method  in 
completing  CPARS  data. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Historically,  the  Department  of  Defense  (DoD)  has  been  the  single  largest 
consumer  of  services  among  government  agencies  (Government  Accountability 
Office  [GAO],  2003).  In  1990,  the  Army’s  spending  alone  on  services  was 
$13  billion  and  grew  to  $74  billion  by  2010  (Ellman,  Livergood,  Morrow,  & 
Sanders,  2011).  From  2000  to  2007,  contract-for-service  spending  remained 
close  to  a  level  percentage  of  total  DoD  spending  until  taking  a  small  dip  in  2008 
(Ellman  et  al.,  2011).  Over  the  past  21  years,  contracting-for-service  spending 
has  grown  in  relation  to  DoD  spending  on  contracting  for  products,  and  was  the 
fastest  growing  area  for  DoD  contract  spending  at  a  growth  rate  of  6.1%  (Ellman 
et  al.,  201 1 ;  see  Figure  1 ). 


Figure  1 .  Service  Contract  Spending  (From  Ellman  et  al.,  201 1 ) 
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In  total  spending  terms,  from  1990  to  2010,  with  the  exception  of  2008,  the 
Army’s  spending  on  contracts  for  services  has  exceeded  the  amount  of  spending 
on  contracting  for  products  (Ellman  et  al.,  2011).  Figure  2  shows  how  Army 
contracting  for  services  has  increased  at  a  much  higher  rate  than  Army 
contracting  for  products  during  this  period.  After  September  11,  2001,  the  Army’s 
total  contract  spending  increased  at  the  fastest  rate  it  had  been  in  21  years 
(Ellman  et  al.,  2011).  Spending  grew  at  139%  for  services,  products,  and 
research  and  development  (R&D)  (Ellman  et  al.,  2011).  After  2008,  contracting 
for  products  decreased  significantly  while  contracting  for  services  has  increased. 
This  growth  in  size  and  scope  of  the  DoD’s  expenditures  on  services  has 
resulted  in  increased  political  attention  in  the  last  few  years.  This  increased 
political  attention  has  allowed  contracting  for  services  to  be  highlighted  and 
scrutinized. 


Figure  2.  Army  Spending  on  Products,  Services,  and  R&D 

(From  Ellman  etal.,  2011) 
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Most  analysts  argue  that  contracting  for  services  is  more  complex  than 
contracting  for  products  or  goods  (Church  &  Schwartz,  2013).  This  difference  in 
complexity  is  because  of  vast  differences  between  the  two  forms  of  contracting. 
According  to  Federal  Acquisition  Regulation  (FAR)  Part  37  (2013),  a  contract  for 
a  service  is  a  contract  in  which  the  contractor  performs  a  recognizable  task  rather 
than  furnishing  an  end  item  of  supply.  Apte,  Apte,  and  Rendon  (2010)  highlighted 
how  services  and  products  contain  different  characteristics.  These  characteristics 
include  “intangibility  of  service  output,  co-production,  simultaneity  of  production 
and  consumption,  the  inability  to  store  services,  and  the  complexity  in  the 
definition  and  measurement  of  services”  (Apte,  et  al.,  2010,  p.  4).  Chairman  of 
the  Defense  Science  Board,  Jacques  Gansler,  published  a  memorandum 
identifying  five  findings  in  the  board’s  assessment  of  service  acquisitions.  One  of 
the  findings  was  that  product  acquisition  is  fundamentally  different  than  the 
acquisition  of  services.  Gansler  stated  that  most  current  regulations,  policies, 
training,  standards,  education,  and  management  structures  are  set  up  to  optimize 
product  acquisition  and  only  a  few  focused  on  the  optimization  of  service 
acquisition  (Gansler,  2011). 

These  differences  have  created  problems  with  contracting  for  services. 
The  DoD  Office  of  Inspector  General  has  stated  that  the  DoD’s  contracting  for 
services  is  not  managed  effectively  and  that  the  “current  process  is 
decentralized,  insufficiently  rigorous,  and  unreliable”  (GAO,  2003,  p.  1).  In 
addition,  the  DoD  Inspector  General  (IG)  has  issued  142  reports  during  Fiscal 
Year  (FY)  2003  to  2008  that  relate  to  the  acquisition  and  contract  administration 
process  (Burton,  2009).  These  reports  listed  deficiencies  that  were  grouped  into 
12  categories  ranging  from  completeness  of  acquisition  support  data  to  material 
internal  control  weaknesses  (Berkheimer,  Burton,  Ford,  Johnson,  Jolliffe, 
Mehiman,  ...  Tucker,  2008).  In  testimony  before  the  Subcommittee  on 
Technology  and  Procurement  Policy  in  2001,  David  Cooper,  GAO  director  of 
acquisition  and  sourcing  management,  identified  several  problems  with 
contracting  for  services  {Contract  Management,  2001).  One  of  the  problems  was 
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that  contracting  officials  were  not  using  prior  experience  of  services  acquisitions 
to  help  define  requirements  more  clearly.  This  led  to  the  use  of  cost 
reimbursement  contracts,  which  placed  the  government  at  risk  for  cost  overruns, 
“despite  39  years  of  past  experience  purchasing  the  same  services”  {Contract 
Management,  2001,  p.  6). 

In  2006,  David  M.  Walker,  serving  at  the  time  as  comptroller  general  of  the 
United  States,  testified  before  the  Subcommittee  on  Appropriations  (DoD 
Acquisitions,  2006).  Walker  testified  that  one  of  the  problems  identified  by  GAO 
with  services  acquisition  was  oversight  of  contracts.  He  testified  that  out  of  90 
service  contracts  reviewed  by  GAO  to  investigate  acquisition  and  contracting 
issues,  nearly  a  third  had  insufficient  contract  oversight.  His  testimony  highlighted 
the  idea  that  the  contracting  office’s  priority  is  to  award  contracts  and  not  to 
ensure  personnel  are  properly  trained  to  conduct  oversight  duties  (DoD 
Acquisitions,  2006).  Without  the  proper  number  of  trained  personnel  for  contract 
oversight  and  the  use  of  poor  contract  arrangements,  the  government  runs  the 
risk  of  not  getting  the  services  it  requests  in  a  timely  manner  and  of  paying  more 
than  is  necessary  (GAO,  2013b).  In  his  memorandum,  Gansler  (2011)  also 
identified  this  weakness  in  his  findings.  He  stated  that  the  workforce  needs 
additional  training,  guidance,  and  experience,  and  that  it  is  inadequately 
prepared  to  acquire  and  execute  service  acquisitions  (Gansler,  2011).  Because 
of  these  problems,  contract  management  has  been  listed  on  GAO’s  High  Risk 
Series  since  1992  and  has  allowed  142  reports  from  the  DoD  IG  on  acquisition 
and  contract  administration  deficiencies  in  five  years  (GAO,  2013b;  Berkheimer 
et  al.,  2008). 

In  response  to  some  of  the  problems  identified,  the  DoD  has  made 
advances  to  improve  service  acquisition.  Some  of  these  changes  aligned  with 
Gansler’s  recommendations,  and  they  are  enforced  by  the  Better  Buying  Power 
(BBP)  2.0  memorandum  from  the  Undersecretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (USD  [AT&L])  Frank  Kendall.  An  example  of  this 
included  the  designation  of  a  senior  manager  for  services  acquisition  in  each 
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DoD  component  (GAO,  2013b).  BBP  2.0  also  mandated  that  each  component 
adopt  a  service  market  segmentation.  This  forced  every  component  in  DoD  to 
adopt  a  standardized  approach  for  categorizing  spending  on  services  to  assist 
with  informed  decision-making  (GAO,  2013b). 

B.  PROBLEM  STATEMENT 

Even  with  the  advancements  in  service  acquisitions,  there  are  still 
problems  with  the  way  service  contracts  are  managed.  These  problems  could  be 
related  to  the  fact  that  there  is  not  an  agreed  upon  definition  of  a  successful 
service  contract  across  the  DoD  (Hagan,  Spede,  &  Sutton,  2012).  The  lack  of  an 
agreed  upon  definition  of  success  has  made  it  difficult  to  measure  a  service 
contract  and  has  made  managing  and  improving  services  aquisition  increasingly 
difficult. 

C.  PURPOSE 

The  purpose  of  this  research  is  to  identify  variables  in  the  service 
contracting  process  that  promote  successful  service  contracts.  The  objective  of 
this  research  is  to  build  on  the  understanding  and  knowledge  that  has  been 
generated  in  past  research  and  use  this  knowledge  to  advance  the  acquisition  of 
service  contracts.  This  research  examines  how  specific  variables  affect  the 
success  of  a  service  contract  in  order  to  improve  services  acquisition 
management  in  the  DoD. 

D.  RESEARCH  QUESTIONS 

The  primary  question  addressed  in  this  research  is,  what  variables  drive 

successful  service  contracts?  This  research  focuses  on  four  research  questions 
that  explore  the  variables  that  result  in  successful  service  contracts.  The 
variables  selected  to  answer  the  research  questions  are  the  types  of  service, 
contractual  amounts,  level  of  competition,  and  contract  type.  This  research 
focuses  on  answering  four  specific  research  questions: 
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1.  Do  the  types  of  services  being  acquired  affect  the  success  of  a 
service  contract? 

2.  Do  the  contractual  amounts  affect  the  success  of  a  service 
contract? 

3.  Does  the  level  of  competition  used  affect  the  success  of  a  service 
contract? 

4.  Does  the  contract  type  affect  the  success  of  a  services  contract? 

The  answers  to  these  questions  determine  whether  or  not  a  relationship 
exists  between  these  four  service  contract  variables  and  the  success  of  the 
contracts.  This  information  will  prove  valuable  in  identifying  variables  that  ensure 
a  successful  service  contract.  The  next  section  will  discuss  the  methodology  of 
this  research. 

E.  METHODOLOGY 

This  research  will  be  performed  by  conducting  a  literature  review  of  the 
service  contract  management  process,  the  contractor  performance  assessment 
reporting  system  (CPARS),  stakeholder  theory,  and  previous  studies  of  service 
acquisition  management.  The  DoD  CPARS  database  will  be  accessed  to  obtain 
data  on  service  contracts  for  specific  service  types.  Statistical  analysis  will  be 
conducted  on  the  CPARS  data  to  draw  conclusions  on  how  certain  variables 
affect  the  success  of  service  contracts.  The  next  section  discusses  the  benefits 
and  limitations  of  this  research. 

F.  BENEFITS  AND  LIMITATIONS 

The  purpose  of  this  research  is  to  identify  variables  in  the  service 
contracting  process  that  promote  successful  service  contracts.  The  benefits  of 
this  research  include  building  on  knowledge  gained  from  previous  research  on 
service  contracting.  It  will  be  the  first  empirical  study  linking  contracting  success 
to  variables  involved  in  contracting  for  services.  This  research  will  give  empirical 
data  showing  how  these  variables  affect  service  contracting.  The  research  will 
identify  variables  that  relate  to  the  success  of  service  contracting. 
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While  the  purpose  of  this  research  is  to  identify  variables  in  the  service 
contracting  process  that  promote  successful  service  contracts,  it  is  not  without  its 
limitations.  The  research  limitations  include  the  analysis  of  contracts  only 
selected  from  the  Army,  contracts  only  submitted  by  Mission  Installation 
Contracting  Commands  (MICCs),  and  contracts  only  from  five  MICC  offices.  The 
second  limitation  is  that  only  contracts  that  relate  to  four  selected  service  types 
were  used  for  the  research.  The  final  limitation  is  the  use  of  CPARS  for  a  proxy 
definition  of  success  and  problems  that  have  been  identified  with  the  CPARS 
process. 

The  first  limitation  is  the  use  of  only  Army  contracts,  only  MICC  contracts, 
and  only  contracts  from  five  MICC  offices  were  chosen  for  the  contracts  to  be 
pulled  from  and  analyzed.  This  limitation  leaves  a  large  area  of  contracting  from 
which  data  was  not  gathered  and  analyzed. 

The  second  limitation  is  that  only  contracts  related  to  four  selected  service 
types  were  chosen  to  be  analyzed.  These  service  types  were  R,  J,  S,  and  D.  R- 
type  contracts  are  contracts  for  professional,  administrative,  and  management 
support  services.  J-type  contracts  contain  maintenance,  repair  and  rebuilding  of 
equipment  type  services.  S-type  contracts  are  contracts  for  utilities  and 
housekeeping  services.  D-type  services  are  for  automatic  data  processing  and 
telecommunications  services  (Apte,  Apte,  &  Rendon,  2012).  This  limitation 
leaves  some  services  from  which  data  was  not  gathered  and  analyzed. 

The  final  limitation  in  this  research  is  the  use  of  the  CPARS  assessment 
as  a  proxy  for  contract  success,  and  the  problems  that  have  been  identified  with 
the  CPARS  process.  The  use  of  the  CPARS  assessment  as  a  proxy  for 
contracting  success  allows  for  stakeholders  to  apply  their  definition  of  success  to 
their  contract.  This  definition  may  be  in  conflict  with  another  stakeholder  involved 
in  the  contracting  process.  This  area  is  covered  in  more  detail  in  the  literature 
review  in  Chapter  II.  The  final  part  of  this  limitation  is  the  problems  that  have 
been  identified  with  CPARS.  This  limitation  includes  agencies  not  reporting  into 

the  database  at  all  or  not  reporting  in  a  timely  manner.  It  also  includes  the 
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limitations  of  agencies  reporting  contract  past  performance  inaccurately  into  the 
federal  database.  This  limitation  is  also  discussed  more  thoroughly  in  Chapter  II. 

G.  SCOPE  AND  ORGANIZATION  OF  RESEARCH 

This  report  is  organized  into  five  chapters.  Chapter  I  contains  the 
background  information,  the  problem  statement,  the  purpose  of  this  research, 
research  questions,  research  methodology,  benefits  and  limitations  of  the  report, 
and  the  scope  and  organization  of  this  research.  Chapter  II  contains  the  literature 
review  of  the  service  contract  management  process,  the  CPARS  system, 
stakeholder  theory,  issues  regarding  CPARS,  and  previous  studies  that  have 
been  conducted  on  service  contract  management.  Chapter  III  provides  a 
methodology  of  how  the  research  was  conducted.  It  describes  how  the  past 
performance  information  was  gathered,  categorized,  and  analyzed.  Chapter  IV 
reveals  the  results  of  the  analysis  and  the  implications  for  DoD  service 
acquisition.  Finally,  Chapter  V  provides  a  summary  and  conclusions,  and 
addresses  areas  for  further  research. 

H.  SUMMARY 

This  chapter  discussed  the  background  information,  the  problem 
statement,  the  purpose  of  this  research,  research  questions,  research 
methodology,  benefits  and  limitations  of  the  report,  and  the  scope  and 
organization  of  this  research.  Chapter  II  contains  the  literature  review  of  the 
service  contract  management  process,  the  CPARS  system.  Stakeholder  Theory, 
issues  regarding  CPARS,  and  previous  studies  that  have  been  conducted  on 
service  contract  management. 


8 


II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  provide  an  overview  of  the  DoD’s  service 
acquisition  process  and  the  current  structure  for  acquiring  these  services.  It 
begins  with  an  examination  of  the  current  service  acquisition  process  and  the 
responsibilities  of  the  stakeholders  involved.  Additionally,  this  chapter  provides  a 
discussion  of  how  a  contractor’s  past  performance  is  recorded  in  the  Contractor 
Performance  Assessment  Reporting  System  (CPARS)  for  use  in  future  source 
selections  and  the  problems  that  have  been  identified  with  the  process.  It  then 
discusses  how  success  may  differ  between  each  stakeholder  and  the  impacts 
that  these  differences  have  on  service  contracts.  Finally,  the  chapter  concludes 
with  a  discussion  of  related  research  that  has  built  a  foundation  for  this  project 
with  a  look  at  six  previous  studies  on  services  acquisition  management  in  the 
DoD. 

B.  SERVICE  CONTRACT  MANAGEMENT  PROCESS 

The  service  contract  management  process  is  distinctly  different  from  the 
product  contract  management  process.  Apte  et  al.,  (2010)  highlighted  how 
services  and  products  contain  different  characteristics.  These  characteristics 
include  “intangibility  of  service  output,  co-production,  simultaneity  of  production 
and  consumption,  the  inability  to  store  services,  and  the  complexity  in  the 
definition  and  measurement  of  services”  (Apte  et  al.,  2010,  p.  4).  Because  of 
these  unique  characteristics,  services  acquisition  must  follow  a  unique  process. 

The  service  acquisition  process  begins  with  the  identification  of  a  service 
requirement  that  is  essential  for  the  organization  to  fulfill  its  mission  (DoD,  2012). 
The  acquisition  process  includes  a  planning  phase,  development  phase,  and 
execution  phase.  The  process  results  in  a  service  that  is  performed  in  a 
prescribed  amount  of  time  and  that  fulfills  the  mission  requirements  (DoD,  2012). 
The  service  acquisition  process  consists  of  seven  steps,  which  are  subtasks  to 
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the  three  phases  introduced  previously  and  shown  in  Figure  3:  plan,  develop, 
and  execute  (DoD,  2012).  Steps  1-3  occur  during  the  planning  phase.  Steps  4 
and  5  take  place  during  the  development  phase,  and  Steps  6  and  7  take  place 
during  the  execution  phase  (DoD,  2012). 
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Figure  3.  The  Services  Acquisition  Process  (From  DoD,  2012) 


The  Planning  Phase 

The  planning  phase  is  the  first  phase  of  the  service  acquisition  process 
and  consists  of  Steps  1,  2,  and  3  (DoD,  2012).  During  this  phase,  the  acquisition 
team  is  formed  and  receives  guidance  from  their  associated  leadership.  The 
acquisition  team  reviews  current  service  contracts  and  the  marketplace  to  create 
a  clear  picture  of  the  specific  needs  the  service  contract  must  fulfill.  (DoD,  2012). 


10 


step  1:  Form  the  Team 

The  acquisition  team  is  formed  during  Step  1  of  the  service  acquisition 
process  (DoD,  2012).  The  purpose  of  this  step  is  to  ensure  that  the  acquisition 
team  is  formed  and  that  the  team  is  provided  the  proper  resources  to  see  the 
contract  to  completion.  This  step  is  further  broken  down  into  six  sub-elements,  as 
seen  in  Figure  4  (DoD,  2012). 


II  THE  PLANNING  PHASE 
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Figure  4.  Planning  Phase  (From  DoD,  2012) 


These  six  sub-elements  ensure  the  formation  of  the  acquisition  team 
(DoD,  2012).  This  begins  with  agency  leadership  support  in  acquiring  the 
required  service.  The  acquisition  team  is  then  officially  formed  and  assigned 
individual  responsibilities.  The  acquisition  team  consists  of  Contracting  Officer, 
Program  Manager,  Contracting  Officer  Representative,  Finance  Manager,  and 
Customer/  End  User.  The  following  paragraphs  explain  the  roles  of  the 
acquisition  team  members. 
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Contracting  Officer 

The  FAR  defines  a  contracting  officer  as  an  individual  with  responsibility 
and  authority  to  obligate  the  government  and  enter  into,  administer,  terminate, 
and  properly  close  out  contracts  (FAR,  2.101).  Contracting  officers  pursue  the 
interest  of  the  United  States  by  ensuring  the  proper  performance  of  all  contracts 
and  enforcing  their  compliance  with  the  statutory  requirements  of  public  policy.  A 
contracting  officer’s  duties  are  not  limited  to  the  solicitation  and  award  of  a 
contract.  The  contracting  officer  retains  contracting  officer  authority  and  the 
associated  supervisory  responsibilities  throughout  the  life  cycle  of  the  contract 
(Apte  &  Rendon,  2007).  Some  DoD  contracting  officers  are  civilian  professionals 
serving  in  1102  positions.  These  1102  civilian  employees  “include  positions  that 
manage,  supervise,  perform,  or  develop  policies  and  procedures  for  professional 
work  involving  the  procurement  of  supplies,  services,  construction,  or  research 
and  development  using  formal  advertising  or  negotiation  procedures”  (Office  of 
Personnel  Management  [0PM],  1983,  p.  3).  A  full  description  of  the  duties  of  the 
contracting  officer  can  be  found  in  FAR  1 .602-2. 

Program  Manager 

The  program  manager  is  responsible  for  overseeing  the  acquisition  team 
and  is  primarily  responsible  for  the  cost,  schedule,  and  performance  of  the 
government  procurement.  Program  managers  make  project  decisions  and  lead 
project  execution  through  effective  collaboration.  They  maintain  continuous 
communication  through  integrated  project  processes  (Apte  &  Rendon,  2007).  The 
program  manager  understands  the  need  of  the  agency  leads  the  market  research 
effort;  and  oversees  the  award,  execution,  and  closeout  of  the  program.  Previous 
research  indicates  that  in  many  service  acquisitions,  a  project  manager  or  project 
team  is  not  assigned  and  the  contracting  officer  assumes  the  responsibilities  of  the 
project  manager  or  project  team  leader  (Apte  &  Rendon,  2007). 
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Contracting  Officer  Representative 

The  contracting  officer  representative  (COR)  is  not  a  contracting -related 
position  and  does  not  have  the  same  statutory  responsibilities  as  the  contracting 
officer  and  program  manager.  The  COR  is  a  subject  matter  expert  who  uses 
technical  expertise  to  supervise  the  execution  of  a  contract  and  provides  periodic 
updates  to  the  contracting  officer.  The  AOR  acts  as  the  “eyes  and  ears”  of  the 
contracting  officer  and  is  a  vital  part  of  the  acquisition  team  (FAR,  2013,  Part  2). 
The  COR  does  not  hold  a  warrant  and  does  not  have  the  authority  to  award, 
alter,  or  terminate  contracts  on  behalf  of  the  U.S.  government.  A  detailed 
description  of  the  responsibilities  of  the  COR  can  be  found  in  FAR  1 .604. 

Finance  Manager 

The  finance  manager  serves  as  a  financial  advisor  to  the  acquisition  team. 
Financial  managers  work  to  ensure  that  the  acquisition  team  is  operating  within 
the  constraints  of  government  authorizations  and  appropriations  in  order  to 
prevent  a  violation  of  fiscal  law  (DoD,  2012).  This  ensures  that  the  acquisition 
team  has  appropriated  funds  available  for  the  contract  and  complies  with  the 
purpose,  time,  and  amount  for  the  contract  in  accordance  with  statutory 
requirements. 

Customer/End  User 

The  customer  serves  as  the  client  to  the  acquisition  team.  The  customer 
must  fully  understand  the  agency’s  needs  and  be  able  to  clearly  convey  those 
needs  to  the  acquisition  team.  This  understanding  allows  the  acquisition  team  the 
ability  to  create  a  cohesive  requirements  document  (DoD  2012). 

Each  of  the  service  contract  stakeholders  have  their  individual  definition  of 
a  successful  service  contract,  which  often  puts  them  in  conflict  with  each  other. 
This  is  discussed  further  in  the  stakeholder  theory  section. 

With  responsibilities  allocated,  the  process  begins  to  move  toward 
understanding  the  stakeholders  and  the  results  they  desire.  During  this  part  of 
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the  process,  individual  stakeholders’  desires  are  consolidated  to  ensure  that  the 
proper  services  are  acquired.  After  the  stakeholders  are  in  agreement  with  each 
other,  the  acquisition  team  develops  a  plan  for  communicating  with  the 
stakeholders  and  the  frequency  of  that  communication.  The  final  parts  of  this 
step  involve  the  creation  of  a  plan  to  document  the  information  gathered  in 
subsequent  steps  and  to  identify  training  requirement  mandates  for  the 
acquisition  team  members  (DoD,  2012). 

Step  2:  Review  Current  Strategy 

Step  2  of  the  service  acquisition  process  is  to  review  the  current 
acquisition  strategy  being  conducted  (DoD,  2012).  This  process  involves 
analyzing  the  requirements  and  desired  results  of  current  service  contracts  and 
attempting  to  draw  parallels  to  the  service  being  acquired.  In  this  step  the 
acquisition  team  attempts  to  gather  all  the  information  necessary  to  develop  a  list 
of  the  exact  requirements  needed  by  the  service.  This  step  contains  10  sub¬ 
elements,  as  seen  in  Figure  5  (DoD,  2012). 
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Step  Two  -  Review  Current  Strategy 
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Figure  5.  Review  Current  Strategy  (From  DoD,  2012) 


These  10  sub-elements  assist  the  acquisition  team  in  developing  a 
thorough  understanding  of  the  service  requirements  (DoD,  2012).  This  process 
begins  with  identifying  current  service  contracts  that  are  similar  to  the 
requirements.  These  services  are  then  analyzed  to  gain  a  better  understanding 
of  the  service  requirements  (DoD,  2012).  The  acquisition  team  determines  the 
risks  that  require  mitigation  and  documents  the  information  gathered  thus  far 
(DoD,  2012).  The  acquisition  team  also  determines  whether  government 
furnished  property  (GFP)  is  used  in  the  current  acquisition  and  determines 
whether  it  is  needed  for  their  service  (DoD,  2012).  The  final  steps  of  this  process 
involve  an  attempt  to  add  a  greater  level  of  detail  to  the  desired  outcomes  and  to 
ensure  the  stakeholders  agree  with  the  acquisition  team  (DoD,  2012). 
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step  3:  Market  Research 

“Market  research  is  required  by  FAR  Part  10  and”  is  essential  for  the 
successful  implementation  of  a  service  contract  (DoD,  2012,  p.  18).  This  step 
involves  gathering  price  and  performance  information  from  the  marketplace 
associated  with  the  service  (DoD,  2012).  A  detailed  understanding  of  the 
market  assists  the  acquisition  team  in  acquiring  the  proper  service  and  avoids 
wasting  taxpayers’  dollars.  There  are  seven  sub-elements  in  this  step,  as  shown 
in  Figure  6  (DoD,  2012). 


Step  Three  -  Market  Research 


Service  Acquisition  Process 
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Figure  6.  Market  Research  (From  DoD,  2012) 


These  seven  sub-elements  ensure  that  the  acquisition  team  has  a  detailed 
understanding  of  current  market  practices  in  the  area  of  the  service  being 
acquired  (DoD,  2012).  The  process  begins  with  dividing  the  information  needed 
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among  the  acquisition  team  members  and  determining  where  the  team  can 
gather  that  information.  The  information  gathered  must  be  collected  in  similar 
formats  across  the  acquisition  team  so  that  it  can  be  compiled  and  disseminated 
appropriately.  The  acquisition  team  members  can  begin  to  contact  service 
providers  after  determining  the  information  that  is  needed.  The  gathered 
information  is  then  analyzed  and  disseminated  throughout  the  team.  The  final 
step  of  this  process  is  to  document  the  collected  market  research,  as  dictated  in 
the  FAR(DoD,2012). 

The  Development  Phase 

Steps  4  and  5-  Requirements  Definition  and  Acquisition  Strategy-  occur 
during  the  development  phase  (DoD,  2012).  This  phase  requires  a  complete 
understanding  of  the  service  objectives  and  how  the  service  will  be  funded  (DoD, 
2012).  During  the  development  phase  the  acquisition  team  gathers  information 
from  the  industry  in  an  effort  to  streamline  the  acquisition  process  and  avoid 
potential  acquisition  problems.  The  final  part  of  this  process  involves  putting  all 
the  gathered  information  together  to  create  an  acquisition  strategy  that 
capitalizes  on  the  information  gathered  in  an  effort  to  obtain  the  correct  service 
for  the  government  (DoD,  2012). 

Step  4:  Requirements  Definition 

Step  4  in  the  service  acquisition  process  involves  compiling  the 
information  gathered  thus  far  and  developing  a  specific  list  of  all  the  requirements 
for  the  service  contract  (DoD,  2012).  This  step  specifically  involves  taking  a 
detailed  look  at  the  individual  requirements  and  understanding  how  they  interact 
with  each  other.  This  step  is  broken  into  the  eight  sub-elements  shown  in  Figure 
7  (DoD,  2012). 
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Figure/.  Requirements  Definition  (From  DoD,  2012) 

These  eight  sub-elements  facilitate  a  complete  understanding  of  the 
requirements  that  the  service  is  going  to  fulfill,  along  with  the  associated  costs 
(DoD,  2012).  The  process  begins  with  a  detailed  understanding  of  the  risks  in 
acquiring  the  service.  Most  service  acquisition  projects  are  analyzed  in  terms  of 
the  likelihood  of  a  given  risk  occurring  and  its  associated  severity.  This 
information  is  often  compiled  on  the  DoD  Risk  Analysis  Model  shown  in  Figure  8 
(DoD,  2012).  Beginning  with  high-risk  areas,  risk  of  contract  failure  or  delay  has 
to  be  identified  and  mitigated  appropriately  (DoD,  2012). 
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Figure  8.  Risk  Analysis  (From  DoD,  2012) 


Once  the  risks  are  analyzed,  the  acquisition  team  begins  to  create  a 
document  describing  how  to  fulfill  and  prioritize  the  service  requirements  (DoD, 
2012).  One  of  the  tools  used  to  complete  this  task  is  the  requirements  roadmap 
tool  shown  in  Figure  9.  The  requirements  roadmap  clearly  defines  and  prioritizes 
the  service  requirements  through  the  duration  of  the  acquisition  process  (DoD, 
2012). 
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Figure  9.  Requirements  Roadmap  (From  DoD,2012) 


The  final  steps  in  this  process  involve  detailing  the  exact  services  required 
by  the  agency  or  customer  and  ensuring  that  the  stakeholders  agree  with  the 
estimated  costs  and  the  prioritization  of  the  requirements  (DoD,  2012). 

Step  5:  Acquisition  Strategy 

The  plan  for  determining  which  type  of  contract  to  use  and  how  the  source 
selection  is  determined  is  part  of  the  acquisition  strategy  (DoD,  2012).  This  step 
involves  putting  together  all  of  the  previously  gathered  information  and  creating  a 
plan  to  achieve  the  contracting  objectives  (DoD,  2012).  This  step  is  broken  down 
into  the  five  sub-elements  seen  in  Figure  10. 
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Step  Five  -  Develop  an  Acquisition  Strategy 
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Figure  10.  Develop  an  Acquisition  Strategy  (From  DoD,  2012) 


The  five  sub-elements  of  the  Acquisition  Strategy  step  begin  with 
determining  the  proper  contract  type  to  use  and  ensuring  that  it  contains  the 
correct  incentives  to  incentivize  the  service  provider  (DoD,  2012).  From  there,  the 
acquisition  team  determines  the  method  that  will  be  used  to  select  the  best  value 
source  for  the  service.  The  acquisition  team  is  assigned  responsibilities  for 
evaluating  the  proposals  received,  and  a  plan  is  created  to  disseminate  the 
information  amongst  the  team.  The  final  step  involves  analyzing  whether  a  draft 
request  for  proposal  (RFP)  should  be  used  in  order  to  obtain  more  feedback  from 
the  industry;  this  step  is  useful  in  reducing  any  misunderstandings  or  confusion 
from  the  industry  and  allows  for  a  smoother  acquisition  process  (DoD,  2012). 
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The  Execution  Phase 


All  the  planning  and  development  conducted  thus  far  are  implemented 
during  the  execution  phase  (DoD,  2012).  This  phase  contains  Steps  6  and  7  and 
begins  with  the  publishing  of  a  solicitation  document.  The  solicitation  document 
officially  communicates  requirements  to  the  marketplace.  From  here,  offerors 
submit  their  proposals,  which  are  evaluated  on  their  own  merit,  and  then  each 
proposal  is  evaluated  for  its  ability  to  meet  the  requirements  listed  in  the 
solicitation  document.  Once  the  contract  is  awarded,  the  contractor  and  the 
government  begin  to  operate  as  a  team  as  opposed  to  individual  entities.  This 
phase  could  eventually  lead  to  the  planning  of  a  follow-on  acquisition  in  the  event 
the  service  is  needed  in  perpetuity  (DoD,  2012). 

Step  6:  Execute  Strategy 

Step  6  marks  the  official  start  to  the  initiation  of  the  service  acquisition, 
from  the  official  request  of  a  service  to  the  selection  of  the  provider  of  that  service 
(DoD,  2012).  As  seen  in  Figure  11,  this  step  includes  seven  sub-elements. 
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step  Six  -  Execute  the  Strategy 
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Figure  1 1 .  Execute  the  Strategy  (From  DoD,  2012) 


The  seven  sub-elements  begin  with  the  official  release  of  the  RFP  (DoD, 
2012)  The  proposals  received  from  the  industry  are  evaluated  based  on  the 
criteria  outlined  in  the  acquisition  strategy  from  Step  5.  Once  the  source  selection 
has  been  determined,  the  companies  that  did  not  win  the  bid  are  debriefed.  This 
debrief  is  intended  to  provide  the  companies  with  information  that  could  help 
them  be  more  competitive  on  future  procurements.  The  final  steps  in  this  process 
solidify  the  quality  assurance  surveillance  plan  (QASP;  DoD,  2012). 

Step  7:  Performance  Management 

Step  7  is  the  final  step  in  the  service  acquisition  process  and  is  essentially 
the  implementation  of  the  awarded  contract  (DoD,  2012).  The  step  involves 
actively  managing  the  contract  to  achieve  the  mission  requirements.  Expectation 
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and  contract  management  are  key  components  of  this  step.  This  step  is  broken 
down  into  the  six  sub-elements  shown  in  Figure  12  (DoD,  2012). 


step  Seven  -  Performance  Management 
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Figure  12.  Performance  Management  (From  DoD,  2012) 


The  sub-elements  in  this  step  begin  with  measuring  the  contractor’s 
performance  as  described  in  the  signed  contract  (DoD,  2012).  The  contract  is 
administered  during  this  step  to  ensure  that  all  parties  are  performing  their 
assigned  duties,  which  includes  compensating  the  contractor  for  providing  the 
accepted  services.  Another  important  part  of  contract  administration  is  regularly 
informing  the  contractor  of  his  or  her  positive  or  negative  performance  of  the 
services  they  are  providing.  Analyzing  and  documenting  the  completed 
acquisition  concludes  the  service  acquisition  process  (DoD,  2012).  During  this 
final  phase  of  the  service  acquisition  process,  the  contractor’s  performance  is 


24 


documented  for  use  in  future  source  selections.  The  past  performance 
information  is  captured  in  a  database  called  the  Contractor  Performance 
Assessment  Reporting  System  (CPARS),  which  is  discussed  in  the  next  section. 

C.  THE  CONTRACTOR  PERFORMANCE  ASSESSMENT  REPORTING 

SYSTEM 

The  FAR  defines  past  performance  as  “an  offeror’s  or  contractor’s 
performance  on  active  and  physically  completed  contracts”  (Part  2).  Past 
Performance  Information  (PPI)  is  “used  to  communicate  contractor  strengths  and 
weaknesses  to  source  selection  officials  and  contracting  officers”  (DoD,  2011,  p. 
50).  CPARS  is  used  in  contracting  to  capture  past  performance  information  (PPI) 
for  use  in  determining  contractor  responsibility  (FAR,  2013,  Part  9.104),  during 
the  source  selection  process  (FAR,  2013,  Part  15)  and  during  the  contract 
administration  process  (FAR,  2013,  Part  42).  This  research  focuses  on  the 
source  selection  process. 

In  1994,  Congress  viewed  past  performance  to  be  relevant  in  source 
selection  and  included  this  in  the  Federal  Acquisition  Streamlining  Act 
(Berkheimer  et  al.,  2008).  In  1998,  the  Navy  created  the  CPARS  to  collect 
historical  information  and  to  be  in  compliance  with  the  congressional  act 
(Berkheimer  et  al.,  2008).  In  2004,  the  undersecretary  of  defense  for  acquisition, 
technology,  and  logistics,  defense  procurement,  and  acquisition  policy 
designated  CPARS  as  the  “solution  for  collecting  contractor  past  performance 
information”  (Berkheimer  etal.,  2008,  p.  i). 

The  Purpose  of  CPARS 

CPARS  was  directed  by  the  DoD  and  is  designed  to  ensure  that  up-to- 
date  information  was  available  on  a  contractor’s  past  performance,  and  is 
intended  to  be  used  in  the  source  selection  process  (DoD,  2011).  “The  CPAR 
assesses  a  contractor’s  performance,  both  positive  and  negative,  and  provides  a 
record  on  a  given  contract  during  a  specified  period  of  time”  (DoD,  2011,  p.  1). 
“The  CPARS  process  is  designed  with  a  series  of  checks-and-balances  to 
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facilitate  the  objective  and  consistent  evaluation  of  contractor  performance.  Both 
Government  and  contractor  perspectives  are  captured  on  the  CPAR  form”  (DoD, 
2011,  p.  2).The  information  contained  in  CPARS  is  designed  to  be  objective, 
though  subjective  data  can  be  used  in  the  event  that  measurement  data  is  not 
identifiable  (DoD,  2011).  Once  CPARS  data  is  entered,  it  goes  into  the  Past 
Performance  Information  Retrieval  System  (PPIRS),  which  collects  past 
performance  information  (DoD,  2012).  It  is  important  to  note  that  the  information 
contained  in  PPIRS  “is  a  method  of  recording  contractor  performance”  and  must 
not  be  the  sole  information  used  in  determining  a  source  selection  (DoD,  2012,  p. 
v).  CPARS  data  entry  is  mandatory  for  any  service  contract  exceeding 
$1,000,000  and  the  entries  are  completed  by  several  different  stakeholders. 

CPARS  Roles  and  Responsibilities 

Five  individuals  are  responsible  for  the  overall  CPARS  input  process:  the 
activity  CPARS  focal  point,  the  assessing  official  representative  (AOR),  the 
assessing  official  (AO),  the  designated  contractor  representative,  and  the 
reviewing  official  (DoD,  2011).  The  activity  CPARS  focal  point  is  responsible  for 
registering  the  contract  into  the  CPARS  database  and  is  the  individual  who 
provides  CPARS  access  authorization.  The  assessing  official  representative 
(AOR)  has  the  responsibility  of  assisting  the  AO  as  needed,  and  provides  a 
narrative  that  contains  information  regarding  the  quality  of  the  service  (DoD, 
2011).  AORs  are  typically  CORs  who  are  often  assigned  from  the  technical, 
functional,  quality  assurance,  specialty,  program  management  or  contracting 
offices  or  the  end  users  of  a  service.  The  assessing  official  (AO)  has  the  overall 
responsibility  of  ensuring  that  the  CPARS  data  and  the  overall  service  is 
completed.  The  AO  may  be  a  program  manager  or  the  equivalent  individual 
responsible  for  the  management  of  the  service  acquisition.  The  AO  reviews  the 
narrative  submitted  by  the  AOR  and  provides  an  overall  evaluation  of  the 
contractor’s  performance.  If  any  of  the  contractor’s  comments  are  modified  by  the 
AO,  the  AO  is  responsible  for  ensuring  that  the  contractor  is  aware  of  them  prior 
to  the  CPARS  process  being  closed  out  (DoD,  2011). 
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The  designated  contractor  representative  is  the  individual  who  receives 
the  completed  evaluations  from  the  AO  (DoD,  2011).  This  representative  then 
reviews,  comments  on,  and  returns  the  evaluation  to  the  AO.  The  designated 
contractor  representative  has  the  ability  to  request  a  reviewing  official  review  if 
deemed  necessary.  The  reviewing  official  has  the  authority  to  provide  a  narrative 
to  the  CPARS  data  and  is  the  individual  responsible  for  signing  the  CPAR,  which 
officially  closes  out  the  CPARS  process.  Once  the  CPAR  is  signed,  the 
information  is  available  through  PPIRS  for  use  by  a  future  source  selection  team 
for  a  period  of  three  years. 

The  quality  of  the  information  in  each  CPARS  data  entry  is  determined  by 
the  quality  of  the  information  provided  by  the  individuals  (DoD,  2011).  Therefore, 
the  effectiveness  of  CPARS  is  determined  by  the  individuals  who  capture  the 
data.  While  the  CPAR  system  has  the  capacity  to  capture  PPI  and  provide 
valuable  information  to  acquisition  officials,  the  system  requires  timely  and 
detailed  information  from  many  different  stakeholders  to  fulfill  its  potential.  This 
often  leads  to  issues  among  the  stakeholders  involved  in  the  service  acquisition 
since  they  all  may  not  have  the  same  objectives  in  the  way  they  assess  the 
contractor’s  performance. 

The  contracting  officer,  program  manager,  CCR,  contractor,  FM,  and 
customer  all  have  different  objectives  in  the  way  each  manages  a  service 
contract.  The  contracting  officer  is  primarily  concerned  with  a  contract’s 
adherence  to  statutory  requirements  and  public  policy,  and  while  required  to 
support  cost,  schedule,  and  performance  objectives  the  Program  Manager  is 
held  solely  accountable  for  their  management  (FAR,  2.101;  DoD,  2012).  The 
COR  technical  expert  would  consider  a  services  contract  successful  if  it  meets  or 
exceeds  technical  requirements  while  not  necessarily  valuing  statutory 
requirements  or  the  cost,  schedule,  and  performance  of  the  contract  (Apte  & 
Rendon,  2007).  The  contractor  values  obtaining  the  highest  profit  possible  while 
satisfying  the  needs  of  the  customer  to  obtain  future  business.  The  financial 
manager  is  solely  concerned  with  ensuring  that  the  acquisition  team  is  operating 
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within  current  appropriations  and  authorizations.  Lastly,  the  customer  is  mainly 
concerned  with  his  or  her  needs  being  met  as  expeditiously  and  efficiently  as 
possible. 

Based  on  the  different  definitions  of  success  by  the  stakeholders,  it  is  easy 
to  see  how  the  stakeholders  are  sometimes  in  conflict  with  each  other.  The 
contracting  officer  may  need  more  time  to  meet  the  statutory  requirements  of  a 
contract,  which  may  adversely  affect  the  program  manager’s  schedule.  The 
financial  manager  may  not  be  able  to  allocate  enough  money  or  even  the  correct 
appropriation  (color  of  money)  to  a  service  to  meet  the  contractor’s  required  cash 
flow.  Lastly,  the  customer  may  value  a  service  provided  but  the  service  may  not 
meet  the  technical  requirements  according  to  the  COR.  Stakeholder  theory  helps 
to  explain  why  these  different  stakeholders  have  different  objectives  and  how  it 
can  assist  in  the  management  of  a  service  contract. 

D.  STAKEHOLDER  THEORY 

Stakeholder  theory  states  that  in  business,  the  stakeholder’s  responsibility 
is  to  generate  as  much  value  as  possible  for  his  or  her  agency  or  business 
(Donaldson  &  Preston,  1995).  The  founding  father  of  stakeholder  theory,  R. 
Edward  Freeman,  assumed  that  the  purpose  of  the  corporation  is  to  generate 
and  dispense  some  form  of  wealth  to  various  stakeholders,  and  that  in  order  to 
achieve  that  purpose,  all  of  the  stakeholders  cooperate  (Freeman,  1984). 
Applying  stakeholder  theory  to  services  contracting  involves  all  members  of  the 
government  acquisition  team,  contractors,  and  the  end  user  of  a  certain  service. 
The  members  of  the  government  acquisition  team  include  the  contracting  officer, 
program  manager,  contracting  officer  representative,  finance  manager,  and 
customer  or  end  users.  The  different  stakeholders  represented  in  this  model 
have  very  different  ways  of  measuring  success  that  are  often  in  conflict  with  each 
other  due  to  their  differing  goals.  When  the  government  and  contractor  negotiate 
a  contract,  the  terms  and  conditions  represent  a  compromise  aimed  at  capturing 
the  best  value  for  the  respective  parties.  These  stakeholders  also  input 
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qualitative  data  based  on  the  performance  of  the  contractor  into  the  CPARS 
database,  which  gives  them  the  opportunity  to  explain  how  they  measure  the 
success  of  a  contract.  Even  with  different  stakeholders  working  together  to 
execute  a  contract,  there  are  still  issues  with  how  the  process  is  captured  in  the 
CPARS  database.  In  addition  to  the  different  stakeholders  and  potential 
conflicting  stakeholder  goals,  there  are  other  issues  and  challenges  involved  in 
managing  PPL 

E.  CURRENT  ISSUES  REGARDING  CPARS 

The  DoD  inspector  general  conducted  a  study  in  2008  on  the  controls  and 
systems  in  place  for  contractor  past  performance  records.  They  found  that 

•  39%  were  registered  more  than  a  year  late  from  contract  award; 

•  68%  had  performance  reports  that  were  overdue;  and 

•  82%  of  past  performance  assessment  reports  reviewed  did  not 
contain  detailed,  sufficient  narratives  to  establish  that  ratings  were 
credible  and  justifiable.  (Berkheimer  etal.,  2008,  p.  i) 

At  the  time  of  this  report,  the  IG  found  that  321  Army  contracts  had  not 
been  registered  in  CPARS  (Berkheimer  et  al.,  2008).  These  contracts  were  not 
being  reported  as  required,  so  there  is  not  any  past  performance  data  in  the 
system  on  these  contracts  (Berkheimer  et  al.,  2008).  The  report  also  found  that 
82%  of  the  performance  assessments  lacked  detailed  narratives  to  support  the 
ratings  given  to  the  contractor  (Berkheimer  et  al.,  2008). 

The  GAO  conducted  another  study  in  2013  in  response  to  two  previous 
studies  in  2009  by  the  GAO  and  a  2011  study  by  the  Office  of  Federal 
Procurement  Policy  (OFPP;  GAO,  2013a).  In  2009,  the  GAO  found  that  contract 
reporting  systems  were  not  complete,  accurate,  and  (due  to  limitations)  effective 
(GAO,  2009).  The  OFPP  reported  that  the  DoD  lacked  sufficient  narratives  to 
justify  ratings  given  to  the  contractors  (GAO,  2013a).  In  2013,  the  GAO  found 
that  PPIRS  compliance  for  timeliness  was  up  from  56%  in  2011  to  74%  in  2013. 
This  shows  that  a  little  over  a  quarter  of  the  required  contracts  are  not  submitted 
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on  time  into  the  reporting  system.  The  DoD  compliance  was  shown  to  be  just 
lower  than  the  average  compliance  of  all  the  services  at  73%  submitted  in  time 
(GAO,  2013a).  DoD  officials  attributed  the  shortcomings  of  the  reporting  system 
to  the  acquisition  workforce  shortages  and  high  turnovers  (GAO,  2013a). 

The  GAO  and  the  DoD  IG  investigations  showed  that,  although  the 
systems  already  in  place  to  evaluate  past  performance  are  improving,  they  are 
still  flawed  (GAO,  2013a).  The  National  Defense  Authorization  Act  (NDAA)  for 
FY2012  requires  a  DoD  strategy  to  ensure  past  performance  assessments  are 
complete,  timely,  and  accurate  (GAO,  2013a).  These  studies  have  shown  that 
the  DoD  still  has  improvements  to  make  to  be  compliant  with  the  NDAA.  Despite 
the  concerns  of  conflicting  stakeholders  and  the  DoD  IG  and  GAO  findings,  the 
use  of  CPAR  assessments  as  a  surrogate  for  contracting  success  is  based  on 
previous  studies  of  service  acquisition,  which  is  discussed  in  the  following 
section. 

F.  PREVIOUS  STUDIES 

This  research  is  a  follow-on  to  a  stream  of  past  research  studies  on 
services  acquisition  in  the  DoD  that  started  in  2006.  Apte,  Ferrer,  Lewis,  and 
Rendon  (2006)  observed  that  the  DoD’s  acquisition  volume  increased  in  the 
decade  prior  to  2006.  They  began  an  initial  exploratory  analysis  in  order  to  frame 
the  DoD’s  service  acquisition  environment.  This  research  revealed  several 
significant  findings  that  impact  service  contracting.  They  noted,  as  mentioned  in 
the  background  section,  the  large  increase  in  service  contracting.  They  stated 
that  service  contracting  has  increased  by  66%  since  1999  (Apte  et  al.,  2006).  In 
2003,  the  DoD  spent  $118  billion,  which  is  approximately  57%  of  the  total  DoD 
procurement  dollars  (Apte  et  al.,  2006).  They  observed  that  four  services  made 
up  approximately  50%  of  total  DoD  spending  on  services  (Apte  et  al.,  2006). 
These  services  were 

•  professional,  administrative,  and  management  support  services; 

•  construction,  repair,  and  maintenance  of  structure  and  facilities; 
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•  equipment  maintenance;  and 

•  information  technology  (IT)  services  (Apte  et  al.,  2006). 

They  observed  that  frequent  communication  was  essential  in  the  success 
of  service  contracts  (Apte  et  al.,  2006).  Apte  et  al.  (2006)  discussed  a  new 
technique  called  the  Business  Requirement  Advisory  Group  (BRAG).  When  used 
successfully,  the  BRAG  facilitates  communication  in  the  administration  of 
contracts  through  the  contract  life  cycle.  This  study  brought  attention  to  the 
centralization  of  service  contracts.  It  showed  the  advantages  and  disadvantages 
of  centralization,  such  as  the  difficulty  of  communication  if  the  contract  is  being 
executed  at  a  different  location  than  where  the  contracting  officer  is  located. 
According  to  Apte  et  al.  (2006),  the  downsizing  of  the  acquisition  workforce  and 
the  push  to  aggressively  comply  with  Circular  A-76  (Office  of  Management  and 
Budget  [0MB],  2003)  contradict  the  importance  of  having  the  correct  number  of 
trained  acquisition  workforce  to  be  successful.  This  compliance  with  the  Circular 
A-76  causes  the  value  for  the  money  to  not  be  realized.  The  most  significant  find 
in  this  research  is  that  while  the  DoD  has  increased  service  contract  spending, 
the  management  infrastructure  has  not  been  maturing  like  product  acquisition 
(Apte  et  al.,  2006). 

In  2007,  Apte  and  Rendon  completed  their  second  exploratory  research 
project  on  services.  This  research  looked  further  into  the  supply  chain 
management  of  service  contracting  and  the  application  of  a  program 
management  infrastructure  (Apte  &  Rendon,  2007).  Their  research  identified  that 
the  traditional  approach  to  managing  the  service  supply  chain  in  the  DoD  and, 
specifically,  the  Air  Force  “does  not  include  a  centralized  and  coordinated 
management”  approach  (Apte  &  Rendon,  2007,  p.  33).  The  Air  Force’s  approach 
does  not  include  a  “life  cycle,  a  designated  program  manager,  cross-functional 
teams,  or  an  enabling  organizational  structure”  (Apte  &Rendon,  2007,  p.  33). 
During  their  research,  Apte  and  Rendon  did  observe  two  types  of  innovative 
approaches  to  this  structural  void  to  assist  the  procuring  agency  in  becoming 
more  successful.  The  first  approach  was  the  Air  Education  and  Training  Center 
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(AETC).  The  AETC  allows  for  a  coordinated  pre-award  management  of  services 
acquisition.  The  AETC  “does  not  maintain  an  on-site  program  manager”  (Apte  & 
Rendon,  2007,  p.  33).  Not  keeping  a  program  manager  on  site  allows  for  the 
system  to  potentially  fail  because  of  broken  communication  between  the 
interested  parties.  The  ACC  (Air  Combat  Command)  model,  which  uses  the 
Acquisition  Management  and  Integration  Center  (AMIC)  approach  to  service 
contracting,  allows  for  a  unique  process  that  provides  a  cradle-to-grave 
acquisition  approach  (Apte  &  Rendon,  2007). 

In  their  third  exploratory  research  study,  conducted  in  2008,  Apte,  Apte, 
and  Rendon  continued  their  research  into  the  management  of  services 
acquisition  at  military  bases  throughout  the  DoD.  This  research  was  a  survey- 
based  look  at  the  Air  Force  and  Navy’s  techniques  for  dealing  with  service 
contracts.  This  research  provided  real  data  on  characteristics  of  service 
contracting,  the  management  approaches  used,  and  the  program  management 
issues  inside  the  DoD  (Apte  et  al.,  2008). 

In  Apte,  Apte,  and  Rendon’s  (2008)  research,  the  Air  Force  covered  four 
service  categories:  professional  management  services,  maintenance,  data 
processing,  and  transportation.  The  researchers  looked  at  the  degree  of 
competition,  contract  type,  and  incentives  used.  The  survey  explored  the  level  of 
services  that  are  acquired  in  the  Air  Force.  The  survey  also  included  questions 
asking  whether  a  project  team  approach  was  used,  who  was  the  leader  of  the 
team,  and  who  owns  the  requirements  for  the  acquisition.  Apte  et  al.  (2008) 
found  that  the  Air  Force  services  they  examined  used  competition  the  majority  of 
the  time. 

The  researchers  found  that  the  Air  Force  used  firm  fixed -price  contracts 
the  majority  of  the  time,  but  rarely  used  incentives,  as  seen  in  Table  1  (Apte  et 
al.,  2008).  The  data  showed  that  the  majority  of  the  service  acquisition, 
throughout  the  life  cycle,  was  conducted  at  the  installation  level  as  opposed  to 
the  regional  level.  The  research  revealed  that  the  contracting  officer  led  the 

project  teams  as  opposed  to  the  program  manager.  This  indicates  that  there  was 
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not  program  management  involvement  in  the  acquisition  process  of  services  at 
the  installation  level  (Apte  et  al.,  2008). 


Table  1 .  Contract  Management  Approach  Found  in  the  Air  Force 

(From  Apte  etal.,  2008) 
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Apte  et  al.’s  (2008)  research  included  the  following  Navy  services: 
professional  and  administrative,  maintenance  and  repair,  data  processing,  and 
utilities  and  housekeeping  (Apte  et  al.,  2008).  The  survey  for  the  Air  Force  was 
replicated  for  the  Navy  in  this  research.  Apte  et  al.  (2008)  covered  six  regions, 
which  encompassed  66  installations  (Apte  et  al.,  2008). 

Apte  et  al.  (2008)  found  that  the  Navy’s  organizational-level  handling  of 
service  contracting  was  different  than  the  Air  Force’s.  The  Navy  acquired  a  little 
over  half  of  its  professional  services  at  the  regional  level  (Apte  et  al.,  2008).  Apte 
et  al.  (2008)  found  that  68%  of  the  Navy’s  data  processing  services  were 
acquired  and  managed  at  the  regional  level.  The  researchers  also  found  that 
approximately  half  of  the  utilities  and  housekeeping  services  were  acquired  at  the 
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regional  level,  and  the  other  half  were  acquired  and  managed  at  the  installation 
level  (see  Table  2;  Apte  et  al.,  2008). 


Table  2.  Results  From  Study  of  Navy  Contracting,  Organization  Level  (From 

Apte  et  al.,  2008) 
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The  data  shows  that  the  Navy  used  competition  a  majority  of  the  time  in  all 
the  four  categories  of  services  (Apte  et  al.,  2008).  The  data  also  revealed  that  the 
Air  Force  contract  characteristics  were  very  similar  to  the  Navy’s.  The  study 
showed  that  a  majority  of  the  time,  firm  fixed-price  contracts  were  used;  however, 
incentives  were  not  used  very  often  (Apte  et  al.,  2008). 

The  Navy  used  a  project  team  approach  only  half  the  time  when  acquiring 
services  (Apte  et  al.,  2008).  Researchers  found  that  when  project  teams  were 
used,  they  were  only  led  by  project  managers  half  of  the  time.  The  research  also 
found  that  when  a  project  team  was  not  used,  either  the  customer  or  contract 
officer  managed  the  acquisition  effort  (Apte  et  al.,  2008). 
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This  research  showed  common  characteristics  between  the  handling  of 
service  contracts  in  the  Navy  and  the  Air  Force,  such  as  the  use  of  firm  fixed- 
price  contracts,  the  use  of  competition,  and  the  minimum  use  of  incentives  (Apte 
et  al.,  2008).  The  research  revealed  no  standard  for  managing  service  contracts 
in  the  DoD.  It  showed  how  the  Air  Force  managed  its  contracts  at  the  installation 
level,  while  the  Navy  conducted  most  of  its  management  from  regional  offices. 
The  study  highlighted  the  different  ways  that  agencies  treat  service  contracts, 
such  as  using  project  teams.  The  research  showed  that,  even  when  project 
teams  were  used,  the  contracting  officer  led  the  teams.  This  trend  has  the 
potential  to  cause  problems  because  the  lines  of  authority  become  grayer,  and 
because  team  personnel  are  not  working  directly  for  the  contracting  officer  (Apte 
et  al.,  2008). 

In  2009,  Apte,  Apte,  and  Rendon  published  their  fourth  study  of  service 
contracting.  This  research  widened  the  scope  of  the  survey  used  in  2008,  which 
focused  on  the  Air  Force  and  Navy,  to  include  the  Army.  This  allowed  the 
researchers  to  gain  a  better  understanding  of  how  the  agencies  in  the  DoD 
manage  their  services  differently.  The  2009  research  made  use  of  the  same 
survey  that  was  used  in  Apte  et  al.’s  (2008)  third  research  project  with  the  Navy 
and  Air  Force.  This  survey  was  distributed  to  eight  contracting  offices  which 
included  40  Army  installations.  There  was  a  75%  response  rate  to  this  survey. 
The  services  that  the  research  included  in  this  study  were  professional  and 
administrative,  maintenance  and  repair  of  equipment,  data  processing  and 
telecommunication,  medical,  maintenance  and  repair  of  equipment,  utilities  and 
housekeeping,  and  transportation  (Apte  et  al.,  2009). 

Apte  et  al.  (2009)  showed  that  the  Army  used  competitively  bid,  firm  fixed- 
price  contracts  without  incentives  in  a  majority  of  the  cases  (see  Table  3).  The 
data  also  showed  that  the  Army  conducted  a  majority  of  its  service  acquisition  at 
the  installation  level.  This  practice  may  have  an  impact  on  the  effectiveness  of 
the  contract  given  the  improved  ability  to  communicate  between  the  interested 
parties  (Apte  et  al.,  2009). 
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Table  3.  Army  Contracting  Results  (From  Apte  et  al.,  2009) 
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The  study  revealed  that  the  Army  used  the  project  team  approach  62%  of 
the  time  (Apte  et  al.,  2009).  The  respondents  said,  however,  that  even  when  a 
project  team  approach  was  used,  the  contracting  officer  led  the  team  61%  of  the 
time  (see  Table  4).  As  mentioned  in  the  2008  Apte  et  al.  research,  this  situation 
could  cause  communication  problems  and  the  graying  of  authoritative  lines 
because  the  contracting  officer  does  not  have  real  authority  over  most  project 
team  members.  Another  finding  of  Apte  et  al.  (2009)  was  related  to  the  use  of  the 
life-cycle  approach;  this  approach  was  used  40%  of  the  time  for  routine  services. 
For  non-routine  services,  the  life-cycle  approach  was  only  used  21%  of  the  time. 
The  use  of  the  life-cycle  approach  should  be  used  to  ensure  proper 
management.  If  services  are  non-routine  in  nature,  this  approach  will  allow  for 
higher  levels  of  uncertainty.  This  uncertainty  and  increased  risk  will  be 
compounded  when  it  is  not  mitigated  through  the  use  of  the  life-cycle  approach 
(Apte  et  al.,  2009). 

This  research  also  found  that  most  acquisition  professionals  disagreed 
that  their  organizations  had  sufficient  positions  and  that  the  available  positions 
were  adequately  filled  (Apte  et  al.,  2009).  The  data  also  showed  that  only  39%  of 
the  respondents  believed  the  acquisition  workforce  was  trained,  and  only  45% 
believed  the  workforce  was  adequately  trained  (Apte  et  al.,  2009). 


36 


Table  4.  Use  of  Project  Team  (From  Apte  et  al.,  2009) 


No.  of 
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61 
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In  2010,  Apte,  Apte,  and  Rendon  completed  their  fifth  research  project. 
This  project  analyzed  the  data  that  was  collected  in  their  previous  studies 
involving  the  Air  Force,  Navy,  and  Army  to  gain  a  better  understanding  of  how 
service  acquisition  was  being  managed  across  the  DoD  (Apte  et  al.,  2010). 

This  research  first  examined  contract  characteristics  for  the  three 
agencies.  The  year  2007  was  the  focus  for  the  research  because  that  year  was  a 
common  time  period  of  data  collected  for  all  three  agencies  (Apte  et  al.,  2010). 
As  seen  in  Figure  13  and  discussed  in  earlier  reports,  full  and  open  competition 
was  used  a  majority  of  the  time. 
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Figure  13.  Contract  Characteristics  Between  Army,  Navy,  and  Air  Force 

(From  Apte  etal.,  2010) 

Apte  et  al.  (2010)  reviewed  the  previous  data  and  compared  the  agencies. 
The  researchers  also  gave  recommendations  to  improve  the  systems  in  place. 
One  such  recommendation  was  for  the  Navy  to  adopt  a  more  disciplined 
approach  and  to  look  at  using  a  possible  virtual  project  team,  since  the 
acquisitions  and  management  mainly  occur  at  the  regional  level  (Apte  et  al., 
2010).  If  this  were  implemented,  the  quality  assurance  evaluator  (QAE)  or 
contracting  officer  representative  (COR)  can  serve  as  the  site  manager,  and 
issues  and  communication  can  improve  between  the  sites  of  the  services  and 
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where  the  contract  is  officially  being  managed  (Apte  et  al.,  2010).  A 
recommendation  for  the  Army  and  Air  Force  was  to  look  at  establishing  an 
installation  project  manager  who  is  overall  in  charge  of  cost,  schedule,  and 
performance  for  services  on  the  installation.  Apte  et  al.  (2010)  recommended  for 
all  of  the  agencies  to  increase  the  effectiveness  of  all  acquisition  training  to 
ensure  a  qualified  workforce. 

Apte,  Apte,  and  Rendon  completed  their  sixth  study  in  2012  on  service 
acquisitions.  This  research  focused  on  the  drivers  of  management  practices  in 
the  Army.  From  the  previous  research  they  conducted,  they  believed  that  four 
factors  had  major  impacts  on  service  contracting  (Apte  et  al.,  2012).  These 
factors  are  the 

•  Type  and  quantity  of  services  being  outsourced  and  the  associated 
acquisition  related  workload. 

•  Characteristics  of  contracts  being  awarded. 

•  Capacity  available  to  carry  out  the  contracting,  project  management,  and 
surveillance  work. 

•  Various  management  practices,  such  as  use  of  project  team  or  life  cycle 
approaches.  (Apte  et  al.,  2012,  p.  6) 

The  relationship  is  shown  in  Figure  14. 


39 


Contract  file  reviews  and  interviews  at  Mission  and  Installation  Contracting 
Commands  (MICCs)  were  conducted  to  complete  this  research  (Apte  et  al., 
2012). 

This  research  focused  on  four  product  service  codes  (PSCs): 

•  R — Professional,  Administrative,  and  Management  Support 
Services; 

•  J — Maintenance,  Repair,  and  Rebuilding  of  Equipment  Services; 

•  S — Utilities  and  Housekeeping  Services;  and 

•  D — Automatic  Data  Processing  and  Telecommunications  Services 
(Apte  et  al.,  2012,  p.  10). 

These  PSCs  account  for  60%  of  the  Army’s  spending  on  services  (Apte  et  al., 
2012).  The  research  focused  on  contract  characteristics  and  management 
practices  (Apte  et  al.,  2012).  These  two  areas  include  the  following: 
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•  Contract  characteristics:  level  of  competition,  contract  type, 
award/incentive  fee,  contract  cost,  number  of  modifications,  award 
basis 

•  Management  Practices:  use  of  Independent  Government  Estimate 
(IGE),  number  of  personnel  assigned,  use  of  a  team  approach, 
acquisition  leadership,  contract  award  time,  acquisition  plan, 
performance  work  statement  or  statement  of  work  (PWS/SOW), 
price  analysis,  price  negotiation  memorandum,  quality  assurance 
plan,  closeout  letter.  (Apte  et  al.,  2012,  p.  10) 

Their  research  showed  some  relationships  between  service  type  and  three 
contracting  characteristics  and  a  statistical  relationship  between  service  type  and 
five  management  practices  (Apte  et  al.,  2012).  These  relationships  are  shown  in 
Figure  15. 


Figure  15.  Relationship  Between  Service  Type,  Contract  Characteristics,  and 

Management  Practices  (From  Apte  etal.,  2012) 

Apte  et  al.  (2012)  found  during  this  project  that  the  average  cost  to  S 
PSCs  was  higher  than  for  the  other  three  service  codes.  They  noted  that  service 
codes  R  and  S  had  a  higher  number  of  modifications  than  D  and  J.  They  found 
that  R  and  S  used  best  value  trade-off  contracts  a  majority  of  the  time,  while  D 
and  J  used  lowest  price  technically  acceptable  (LPTA)  50%  of  the  time.  They 
observed  that  firm  fixed-price  contracts  were  always  used,  but  only  once  with 
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incentives.  They  also  did  not  find  any  relationship  with  the  solicitation  process 
and  service  type  (Apte  et  al.,  2012). 

After  evaluating  the  14  management  practices,  the  researchers  found  five 
management  practices  which  had  a  statistical  relationship  with  the  service  type 
(Apte  et  al.,  2012).  They  also  found  that  if  the  contract  was  over  the  simplified 
acquisition  threshold,  the  contracts  only  had  IGEs  32%  of  the  time.  Apte  et  al. 
(2012)  found  that  the  number  of  people  assigned  to  PSCs  R  and  S  were  higher 
than  for  the  other  service  codes.  Apte  et  al.  documented  that  the  contracting 
officer  led  the  acquisitions  of  the  services  in  146  out  of  the  154  contracts  that 
they  researched,  and  the  contracting  specialist  led  acquisitions  for  the  other  eight 
contracts.  The  researchers  also  stated  that  only  43%  of  the  contracts  they 
reviewed  had  a  QASP  (Apte  et  al.,  2012). 

While  investigating  whether  the  MICC’s  capacity  to  conduct  acquisition 
work  affects  its  management  practices,  Apte  et  al.  (2012)  found  that  there  is  a 
statistical  relationship  between  capacity  and  management  practices.  They  found 
that  the  offices  did  not  have  the  required  number  of  authorized  people  to  conduct 
acquisition  functions.  They  also  found  that  MICC  personnel  lacked  the  proper 
training  certifications  (see  Table  5). 


Table  5.  Contracting  Workforce  Results  (From  Apte  et  al.,  2012) 
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The  researchers  found  that  the  MICCs  have  a  substantial  workload  with 
minimal  personnel,  training,  and  experience  (Apte  et  al.,  2012). 

In  2012,  Hagan,  Spede,  and  Sutton  conducted  research  on  how  the  Navy 
stakeholders  determined  and  defined  success  in  service  contracting.  After 
surveying  168  key  stakeholders,  Hagen  et  al.  found  that  all  the  stakeholders 
tended  to  use  outcome  related  factors  as  opposed  to  process  related  factors 
when  asked  to  define  and  measure  the  success  of  service  contracts.  They 
recommended  that  Navy  service  contract  success  should  be  defined  in  terms  of 
both  the  outcomes  and  processes.  Outcome  success  was  defined  and  measured 
in  terms  of  the  cost,  schedule,  and  performance  of  successful  service  contracts. 
Cost  outcomes  referenced  cost  overruns,  total  profits,  cost  reasonableness,  and 
cost  fairness.  Schedule  related  to  the  ability  to  meet  assigned  timelines  and 
complete  the  contract.  Performance  dealt  with  stakeholder  satisfaction  and  the 
ability  to  perform  the  statement  of  work  (Hagan  et  al.,  2012). 

Process  success  was  defined  in  terms  of  how  the  contract  was 
administered  and  planned  and  the  number  of  protests  it  received  for  the  definition 
of  success  (Hagan  et  al.,  2012).  The  measurement  of  process  success  was 
determined  by  the  communication  between  stakeholders  and  the  number  of 
modifications  the  contract  received.  Administration  dealt  with  the  total  workload 
on  the  acquisition  team  and  the  number  of  contracts  the  contracting  officer 
currently  managed.  Planning  was  related  to  the  ability  of  a  contracting  officer  to 
turn  requirements  into  contractual  obligations.  Number  of  protests  and  number  of 
modifications  were  the  quantitative  numbers  associated  with  each. 
Communication  assessed  the  amount  of  information  flow  and  the  quality  of 
communication  between  the  stakeholders.  A  diagram  of  Hagan  et  al.’s  (2012) 
research  questions  is  shown  in  Figure  16. 
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Figure  16.  Diagram  of  Survey  Questions  (From  Flagan  et  al.,  2012) 


Therefore,  based  on  the  already  mentioned  previous  studies,  this  research 
uses  the  Contractor  Performance  Assessment  Reporting  System  (CPARS)  data 
results  as  a  surrogate  definition  of  a  successful  service  contract,  considering  no 
definitive  definition  has  been  established  by  the  DoD,  and  the  CPARS  captures 
the  outcomes  for  both  defining  and  measuring  a  successful  service  contract  by 
Flagan,  Spede  and  Sutton  (Flagan  etal.,  2012). 

G.  SUMMARY 

This  chapter  contains  a  review  of  the  available  literature  on  DoD  service 
contracting  processes.  This  chapter  presented  an  overview  of  the  service 
contract  management  process  by  providing  an  in-depth  description  of  the  seven 
steps  and  associated  sub-elements.  The  review  of  the  CPARS  database  in  this 
chapter  looked  at  the  information  that  CPARS  provides  to  the  user  and  who  is 
responsible  for  capturing  CPARS  data  throughout  the  service  contract 
management  process.  Next,  the  chapter  described  how  stakeholder  theory 
explains  how  the  differing  goals  of  the  different  stakeholders  can  lead  to  a 
contract  that  provides  the  best  value  to  all  parties.  The  current  issues  facing 
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CPARS  were  also  described  to  highlight  some  of  the  current  problems  the 
system  is  facing.  Previous  studies  relating  to  service  acquisitions  were  also 
explained  to  provide  context  for  why  CPAR  assessments  can  be  used  as  a 
surrogate  for  contract  success.  The  next  chapter  examines  the  research 
questions  for  this  project  and  provides  a  description  of  how  the  research  will  be 
conducted,  the  data  used  in  the  research,  and  the  type  of  analysis  used  in  the 
project. 
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III.  RESEARCH  METHODOLOGY 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  discuss  how  this  research  is  conducted 
and  how  a  successful  service  contract  is  defined  using  the  CPARS  data.  This 
chapter  begins  by  discussing  why  the  CPARS  database  is  used  in  deriving  a 
definition  of  a  successful  service  contract.  It  then  expounds  upon  what  data  is 
retrieved  from  the  CPARS  database  and  how  that  data  is  analyzed.  The  results 
of  the  research  are  revealed  in  Chapter  IV. 

B.  DEFINING  A  SUCCESSFUL  SERVICE  CONTRACT 

CPARS  data  will  be  used  as  a  surrogate  definition  of  a  successful  service 
contract  because  it  captures  the  outcomes  defined  by  Hagan,  Spede,  and  Sutton 
for  both  measuring  and  defining  a  service  contract  (Hagan  et  al.,  2012).  Quality, 
schedule  and  cost  control  assessments  are  captured  in  the  CPARS  database 
and  report  key  aspects  of  a  contractor’s  performance  on  the  contract.  The  Quality 
rating  contains  the  AQ’s  assessment  of  the  contractor’s  qualitative  performance 
and  compares  it  to  the  requirements  stated  in  the  contract.  The  Schedule  rating 
contains  an  assessment  of  the  contractor’s  ability  to  meet  schedules  outlined  in 
the  contract  such  as  “task  orders,  milestones,  delivery  schedules,  and 
administrative  requirements”  (DoD,  2011,  p.  A3-7).  The  Cost  Control  rating 
contains  an  assessment  of  the  contractor’s  ability  to  “forecast,  manage, 
and  control  the  costs”  associated  with  conducting  their  services  (DoD,  2011,  p. 
A3-7).  The  Business  Relations  rating  contains  an  assessment  of  the  contractor’s 
ability  to  coordinate  their  business  activities  such  as  corporate  behavior, 
customer  satisfaction,  management,  and  attitude  towards  customers.  The 
Management  of  Key  Personnel  rating  contains  an  assessment  of  the  contactors 
ability  to  maintain  qualified  individuals  in  key  positions  as  outlined  in  the  contract. 
The  Utilization  of  Small  Business  rating  contains  an  assessment  of  the 
contractors’  ability  to  integrate  small  businesses  in  the  execution  of  the  contract. 
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These  ratings  are  completed  and  approved  in  the  CPARS  database.  Once  the 
CPARS  ratings  are  approved,  the  information  is  stored  and  accessed  through  the 
PPIRS-RC  database.  This  research  obtained  the  CPARS  data  by  accessing  the 
PPIRS-RC  database  (DoD,  2011). 

C.  DESIGNING  THE  DATABASE 

The  search  of  the  PPIRS-RC  website  was  limited  to  the  Army  MICC  non¬ 
system  business  sector.  From  the  results  of  the  search,  a  spreadsheet  was 
created  of  all  the  contracts  contained  within  the  PPIRS-RC  database.  This 
search  yielded  14,395  contracts  in  total,  with  periods  of  performance  end  dates 
ranging  from  1996  through  2013.  The  data  was  then  narrowed  to  only  those 
contracts  containing  a  Product  or  Service  Code  or  Federal  Service  Code 
(PSC/FSC)  of  R,  D,  J,  or  S,  which  yielded  5,621  contracts.  It  is  important  to  note 
that  construction  contracts  were  not  included  in  this  database  since  they  are 
categorized  as  a  distinct  business  sector  within  the  PPIRS-RC  database.  The 
data  was  further  narrowed  by  only  searching  five  contracting  organizations. 
These  organizations  were  Fort  Eustis  (W911S0),  Fort  Knox  (W9124D),  Fort 
Hood  (W91151),  Fort  Bragg  (W91247),  and  Fort  Sam  Houston  (W911SE).  These 
organizations  were  selected  because  they  are  organized  as  the  MICC  Field 
Directorate  Cffices  (FDCs)  of  five  different  areas  of  focus,  as  seen  in  Figure  17. 
This  narrowed  the  data  to  715  distinct  service  contracts  that  were  used  in 
conducting  analysis,  as  seen  in  Table  6.  For  each  contract,  data  was  collected  on 
the  awarded  dollar  value,  basis  of  award,  type  of  contract,  quality  of 
product/service  rating,  schedule  rating,  cost  control  rating,  business  relations 
rating,  management  of  key  personnel  rating,  and  utilization  of  small  business 
rating  in  order  to  answer  the  four  primary  research  questions. 
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Figure  17.  Contracting  Lines  of  Operation  (From  Vollmecke,  2012) 
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Table  6.  Database  Breakdown 


T otal  Contracts 

Total  Army  MICC  Non- 
System  Contracts 

14395 

Removed  Non  R,  J,  S,  D 
Service  Contracts 

8774 

Total  R,  J,  S,  D  Service 
Contracts 

5621 

Removed  other  MICC  R,  J, 
S,  D  Service  Contracts 

4906 

MICC  FDO  Eustis,  Knox, 
Hood,  Bragg,  Sam 

Houston  Service 

Contracts 

715 

Fort  Eustis 

238 

Fort  Knox 

119 

Fort  Hood 

114 

Fort  Bragg 

55 

Fort  Sam  Houston 

189 

Once  the  database  was  established,  it  was  further  refined  by  labeling  each 
contract  as  a  success  or  a  failure.  Labeling  a  contract  as  a  failure  was 
determined  by  whether  a  contract  received  a  marginal  or  unsatisfactory  rating  in 
quality  of  product/service,  schedule,  cost  control,  business  relations, 
management  of  key  personnel,  or  utilization  of  small  business  rating.  Receiving  a 
marginal  or  unsatisfactory  rating  in  any  one  of  these  ratings  labels  the  entire 
contract  as  a  failure.  Those  contracts  not  determined  to  be  failures  were  labeled 
as  successful  service  contracts.  See  appendix  A  for  more  details. 

The  data  obtained  from  PPIRS-RC  was  exclusively  related  to  outcome 
successes  defined  by  Hagan,  Spede,  and  Sutton  (2012).  In  addition  to  the 
PPIRS-RC  outcome-related  data,  organizational  data  was  collected  in  the  areas 
of  annual  workload  in  dollars,  number  of  actions  completed  by  the  MICC,  number 
of  1102  billets  authorized,  and  percent  of  1102  billets  filled.  This  data  was  added 
to  the  database  in  an  effort  to  analyze  some  of  the  process  success 
characteristics  defined  by  Hagan,  Spede  and  Sutton  (Hagan  et  al.,  2012).  Once 
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the  database  was  completed,  it  was  analyzed  to  answer  the  four  research 
questions.  The  next  section  describes  how  the  database  is  analyzed  to  answer 
the  research  questions. 

D.  ANALYZING  THE  DATABASE 

Determination  of  whether  the  types  of  services  being  acquired  affected  the 
success  of  the  contract  was  conducted  by  determining  the  total  number  of 
contracts  in  each  of  the  R,  J,  S,  and  D  type  services  and  the  total  that  were 
determined  to  be  failures.  The  total  failures  for  each  type  of  service  were  divided 
by  the  total  number  of  the  contracts  for  that  type  of  service.  This  result  was  the 
failure  rate  for  that  specific  type  of  service.  This  analysis  revealed  whether  one 
type  of  service  resulted  in  a  higher  failure  rating  then  other  types  of  services. 
These  results  were  then  analyzed  to  determine  the  stated  reasons  that  the 
contracts  were  given  a  marginal  or  unsatisfactory  rating  in  order  to  determine 
whether  a  failure  pattern  emerged  for  the  types  of  services  being  acquired. 
Finally,  additional  analysis  was  conducted  to  attempt  to  identify  other  data 
correlations  that  may  highlight  why  a  certain  service  type  was  evaluated  as  a 
failure.  These  results  are  useful  in  determining  the  management  levers  that  might 
assist  in  ensuring  a  successful  service  contract. 

Determination  of  whether  contractual  amounts  affected  the  success  of  a 
service  contract  was  conducted  in  the  same  manner  as  the  types  of  services 
being  acquired.  The  contract  categories  in  this  analysis  were  broken  down  into 
five  separate  categories:  0-$1  million,  greater  than  $1  million-$10  million, 
greater  than  $10  million-$50  million,  greater  than  $50  million-$1  billion,  and 
greater  than  $1  billion.  The  failure  rates  were  then  determined  for  each  of  these 
categories  and  whether  a  correlation  existed  between  the  ratings  that  the 
contracts  received.  Each  category  then  received  an  in-depth  analysis  of  why  it 
received  a  failure  rating  by  showing  which  category  (quality,  schedule,  cost 
control,  business  relations,  management  of  key  personnel,  and  utilization  of  small 
business)  received  a  marginal  or  unsatisfactory  rating. 
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To  answer  the  third  question  about  whether  the  level  of  competition  used 
affects  the  success  of  a  service  contract,  the  contracts  were  separated  into  three 
different  categories:  competitive,  non-competitive,  and  other.  Failure  rates  were 
used  for  each  of  these  categories  to  reveal  whether  the  level  of  competition  used 
was  related  to  the  failure  of  that  level  of  competition.  The  ratings  of  the 
unsuccessful  contracts  were  then  analyzed  to  determine  which  area  (quality, 
schedule,  cost  control,  business  relations,  management  of  key  personnel,  and 
utilization  of  small  business)  was  more  frequently  associated  with  the  contract 
being  rated  as  marginal  or  unsatisfactory. 

The  final  research  question  focused  on  the  type  of  contract  used  and  the 
success  of  the  contract.  This  required  dividing  the  contracts  into  six  different 
categories.  The  first  four  categories  are  CPAF,  CPFF,  CPIF,  and  FFP.  The 
remaining  two  categories  are  a  combination  and  other.  The  combination  category 
contains  contracts  that  used  a  combination  of  CPAF,  CPFF  and  FFP  together 
instead  of  using  just  one  type.  The  other  category  includes  all  the  contracts  that 
did  not  fit  into  the  previous  five  categories,  such  as  labor  hours  or  time  and 
materials.  The  failure  rates  and  associated  reasons  determined  whether  the 
types  of  contracts  being  used  affected  the  success  of  a  service  contract. 

Once  the  primary  research  questions  were  analyzed,  analysis  on  the 
location,  duration,  year  the  contract  was  completed,  and  the  MICC  organizational 
data  was  used  to  determine  whether  any  of  these  additional  factors  relate  to  the 
four  contract  variables.  The  MICC  organizational  data  focuses  on  four  main 
areas:  annual  workload  in  dollars  spent,  number  of  completed  contracts,  number 
of  authorized  1102  billets,  and  the  percent  of  1102  billets  filled.  This 
organizational  data  proved  useful  in  adding  some  of  the  process  success 
variables  defined  by  Flagan,  Spede  and  Sutton  to  the  analysis  (Flagan  et  al., 
2012).  The  failure  rates  and  rating  patterns  among  these  contracts  provided 
additional  insight  into  the  four  primary  research  questions. 

Answering  these  research  questions  with  the  data  collected  reveals 

whether  the  four  contract  variables  had  a  statistically  significant  impact  on  the 
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success  or  failure  of  a  service  contract.  The  results  of  that  analysis  help  focus 
further  research  to  explain  why  that  is  the  case  and  what  the  DoD  can  do  to 
increase  the  success  rate  of  service  contracts. 

Figure  18  illustrates  the  research  methodology  previously  described.  The 
column  on  the  right  contains  the  six  CPARS  data  areas.  These  areas  are  used  to 
label  each  contract  as  a  success  or  a  failure  based  on  the  ratings  (marginal  or 
unsatisfactory).  The  contracts  labeled  as  failures  are  analyzed  using  the  four 
contract  variables  shown  on  the  left  column.  The  purpose  is  to  determine  if  there 
is  a  relationship  between  contract  variables  and  contract  success. 


Contraa  Variables 


CPARS  Data 


Figure  18.  Research  Methodology 


E.  SUMMARY 

This  chapter  reviewed  the  research  methodology  used  in  answering  the 
four  research  questions.  It  began  by  defining  a  successful  service  contract 
according  to  this  research.  The  chapter  then  described  how  the  database  was 
designed  to  capture  the  data  that  was  analyzed.  Finally,  the  chapter  discussed 
how  the  research  questions  were  answered.  The  next  chapter  shows  the  data 
analysis  and  the  results  of  the  research. 
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IV.  ANALYSIS  AND  IMPLICATIONS 


A.  INTRODUCTION 

This  chapter  reveals  the  results  of  the  research  into  “what  variables  drive 
successful  service  contracts?”  This  chapter  begins  with  an  overview  of  the 
database  created  for  this  report  in  order  to  create  a  clear  picture  of  how  the 
results  were  derived.  This  overview  includes  the  total  number  of  successful  and 
unsuccessful  contracts  separated  into  specific  categories.  Once  the  database  is 
framed,  the  chapter  explores  the  four  primary  research  questions  driving  this 
study,  which  focus  on  the  failure  rates  of  the  different  categories.  The  chapter 
then  explores  additional  information  obtained  from  the  database,  including  the 
Mice  organizational  data,  in  order  to  add  additional  insight  into  the  four  primary 
variables.  The  additional  analysis  enhances  the  understanding  of  the  drivers  of 
success  and  adds  valuable  insight  into  what  makes  service  contracts  successful. 
This  chapter  concludes  with  a  significance  test  and  the  implications  of  the 
findings. 

B.  OVERVIEW  OF  THE  DATA 

The  database  consisted  of  715  contracts  pulled  from  the  PPIRS-RC 
database.  These  contracts  were  from  five  different  MICC  Regional  offices  and 
consisted  of  R  (Professional,  Administrative,  and  Management  Support 
Services),  J  (Maintenance,  Repair,  and  Rebuilding  of  Equipment  Services),  S 
(Utilities  and  Housekeeping  Services),  and  D  (Automatic  Data  Processing  and 
Telecommunications  Services)  type  of  services.  The  data  pulled  represent 
population  data  that  meet  the  criteria  of  the  MICC  Regional  offices  and  the 
services  previously  listed.  Labeling  a  contract  as  a  failure  was  determined  by 
whether  a  contract  received  a  marginal  or  unsatisfactory  rating  in  quality  of 
product/service,  schedule,  cost  control,  business  relations,  management  of  key 
personnel,  or  utilization  of  small  business  rating.  Receiving  a  marginal  or 
unsatisfactory  rating  in  any  one  of  these  ratings  labels  the  entire  contract  as  a 
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failure.  Those  contracts  not  determined  to  be  failures  were  labeled  as  successful 
service  contracts. 

The  data  consisted  of  715  contracts;  22  resulted  in  a  failure  label  as  just 
described  and  as  explained  in  Chapter  III.  This  resulted  in  a  total  contract  failure 
rate  of  3.08%.  These  results  are  clearly  seen  in  Figure  19  and  Table  7. 


Total  Successes  and  Failures 

22 

3.08% 

iJ  Successes 

W  Failures 

693  ' 

96.92S^ 

/ 

Figure  19.  Total  Contracts 


Table  7.  Total  Contract  Information 


Failures 

Success 

Total 

Failure 

Rate 

Contracts 

22 

693 

715 

3.08% 

Contracts  evaluated  and  entered  in  CPARS  are  evaluated  by  quality, 
schedule,  cost  control,  business  relations,  management  of  key  personnel,  and 
utilization  of  small  business  (discussed  in  more  detail  in  Chapter  III).  The  total 
contract  failures  for  each  area  of  evaluation  can  be  seen  in  Figure  20.  Each  area 
of  evaluation  was  given  a  score  respective  of  the  rating  the  evaluator  gave  that 
area  (see  Table  8).  The  areas  of  evaluation  scores  were  then  averaged  to 
examine  what  areas  where  rated  higher  throughout  the  population  (see  Table  9). 
Business  relations  had  the  highest  average  among  the  other  evaluation  areas 
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with  an  average  score  of  3.  Management  of  key  personnel  had  the  next  highest 
average  with  a  score  of  2.68.  Quality,  schedule,  and  utilization  of  small  business 
had  equal  average  scores  of  2.5.  Cost  control  had  the  lowest  average  score  of  all 
the  areas  with  a  2.31  average  score. 


Table  8 .  Area  of  Contract  Eva 

uation  Scores 

Rating 

Score 

Exceptional 

5 

Very  Good 

4 

Satisfactory 

3 

Marginal 

2 

Unsatisfactory 

1 

Table  9.  Average  Success  and  Failure  Rates  of  PPIRS  Areas  of  Contract 

Evaluation 


Successes 

Average 

Ratings 

Failures 

Average 

Ratings 

Quality 

4.19 

2.5 

Schedule 

4.19 

2.5 

Cost  Control 

4.1 

2.31 

Business  Relations 

4.17 

3 

Management  of  Key  Personnel 

4.18 

2.68 

Utilization  of  Small  Business 

4.07 

2.5 
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C.  ANSWERING  THE  RESEARCH  QUESTIONS 

1 .  Research  Questions  Restated 

The  primary  question  addressed  in  this  research  concerns  what  variables 
drive  successful  service  contracts.  This  research  focuses  on  four  research 
questions  that  explore  the  variables  that  result  in  successful  service  contracts. 
These  questions  are  as  follows; 

•  Do  the  types  of  services  being  acquired  affect  the  success  of  a 
service  contract? 

•  Do  the  contractual  amounts  affect  the  success  of  a  service 
contract? 

•  Does  the  level  of  competition  used  affect  the  success  of  a  service 
contract? 

•  Does  the  contract  type  affect  the  success  of  a  service  contract? 

This  information  proves  valuable  in  identifying  the  variables  that  ensure  a 
successful  service  contract. 
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2.  Type  of  Service 

The  first  research  question  is  as  follows:  Do  the  types  of  services  being 
acquired  affect  the  success  of  a  service  contract?  To  answer  this  question,  four- 
product  service  codes  were  examined  using  data  gathered  from  the  PPIRS 
database.  Contracts  with  a  product  service  code  of  R  (Professional, 
Administrative,  and  Management  Support  Services),  J  (Maintenance,  Repair,  and 
Rebuilding  of  Equipment  Services),  S  (Utilities  and  Housekeeping  Services),  and 
D  (Automatic  Data  Processing  and  Telecommunications  Services)  were  analyzed 
to  include  the  failure  rates  of  each  service  contract.  The  database  contained  331 
R-type  contracts,  58  J-type  contracts,  292  S-type  contracts  and  34  D-type 
contracts  (see  Table  8). 

Finding  1:  The  S-type  contracts  had  the  highest  faiiure  rate  of  all  the 
product  service  codes  analyzed. 

The  11  labeled  failures  of  S-type  contracts  resulted  in  a  3.77%  failure  rate 
(see  Figure  21).  There  were  two  reasons  that  tied  for  the  most  common  reasons 
for  S-type  contract  failures.  These  reasons  were  six  business  relation  failures 
and  six  failures  due  to  management  of  key  personnel  (see  Figure  22).  R-type 
contracts  had  nine  labeled  failing  contracts  out  of  331,  giving  R-type  contracts  a 
failure  rate  of  2.72%  (see  Figure  21)  which  was  the  second  lowest.  The  most 
common  reason  entered  into  the  PPIRS  database  was  quality.  J-type  contracts 
consisted  of  58  contracts  with  two  labeled  failures.  This  gave  the  J-type  contracts 
a  failure  rate  of  3.45%.  For  both  labeled  failures  of  the  J-type  contract, 
scheduling  was  listed  as  a  reason  for  both  of  these  failures.  There  were  only  34 
of  D  service  code  contract  types  with  0  failures  (see  Figure  21  and  22). 


Table  10.  Type  of  Service  Acquired  Total  Successes  and  Failures 


Type  of  Service 
Acquired 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

D 

34 

0 

34 

0.00% 

J 

56 

2 

58 

3.45% 

R 

322 

9 

331 

2.72% 

S 

281 

11 

292 

3.77% 

Total 

693 

22 

715 

3.08% 

59 


4.00% 
3.50% 
3.00% 
I  2.50% 

4,  2.00% 

u 

3 

n  1.50% 

u. 

1.00% 

0.50% 

0.00% 


3.77% 


3.45% 


2.72% 


y  IJ 


0.00% 


I  S 

Tj’pe  of  Service  Acquired 


Figure  21.  Failure  Rates  of  the  Different  Product  Service  Code  Contracts 
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Figure  22.  Reasons  for  Product  Service  Code  Failure 


3.  Contractual  Amounts 

The  second  research  question  is  as  follows;  Do  the  contractual  amounts 
affect  the  success  of  a  service  contract?  To  answer  this  question  the  research 
grouped  the  contracts  into  dollar  amounts.  The  different  dollar  amount  categories 
contracts  that  were  0-$1  million,  contracts  greater  than  $1  million-  $10  million, 
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contracts  greater  than  $10  million-$50  million,  contracts  greater  than  $50 
million-$1  billion,  and  contracts  greater  than  $1  billion  (see  Table  11).  The 
researchers  looked  at  these  different  dollar  amounts  separately  and  examined 
the  failure  rate  of  each  group. 

Finding  2:  The  contract  grouping  that  consisted  of  contracts  worth  a 
dollar  amount  greater  than  $50  million-$1  billion  had  the  highest  failure 
rate  of  all  the  groups  in  the  analysis. 

This  group  consisted  of  92  contracts  with  eight  labeled  failures,  giving  the 
group  a  failure  rate  of  8.7%  (see  Figure  23).  This  group’s  most  common  reason 
for  failing  was  cost  control  (see  Figure  24).  This  reason  was  listed  for  six  failed 
contracts.  This  failure  rate  is  much  higher  than  the  total  contract  average  failure 
rate  of  3.08%.  In  the  first  group  that  consisted  of  contracts  that  were  worth  0-$1 
million,  there  were  a  total  of  35  contracts.  In  this  first  group  there  was  only  one 
labeled  a  failure  (see  Figure  23).  This  gave  this  group  a  2.86%  failure  rate.  This 
contract  was  labeled  a  failure  because  of  quality  (see  Figure  24).  The  group 
consisting  of  contracts  greater  than  $1-  50  million  was  the  largest  of  all  the 
grouped  dollar  amounts.  It  consisted  of  466  contracts,  and  of  those,  10  were 
labeled  failures.  That  gave  this  group  a  2.15%  failure  rate  (see  Figure  23).  While 
this  group  had  the  most  failures  numerically,  it  still  was  under  the  average  failure 
rate  because  of  the  amount  of  contracts  total  in  this  group.  The  most  common 
reason  for  this  group  to  fail,  according  to  PPIRS,  was  for  quality,  it  was  cited 
seven  times.  The  contracts  greater  than  $10  million-$50  million  group  consisted 
of  118  contracts.  There  were  three  labeled  failures  in  this  group.  This  group  had 
a  2.54%  failure  rate.  This  group  was  also  under  the  average  total  contract  failure 
rate  of  3.08%.  This  contract  group  most  commonly  failed  for  scheduling  issues 
and  management  of  key  personnel.  It  failed  for  scheduling  twice  and 
management  of  key  personnel  twice  (see  Figure  24).  This  means  that  one  of  the 
contracts  in  this  group  had  both  issues  listed  as  reasons  for  failure.  The  group 
consisting  of  contracts  worth  greater  than  $1  billion  was  the  smallest  group  in  the 
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contractual  amount  grouping.  It  consisted  of  only  four  contracts  and  did  not 
contain  any  labeled  failures. 


Table  1 1 .  Contract  Amount  Total  Successes  and  Failures 


Contract  Dollar 
Amount 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

$0-$1M 

34 

1 

35 

2.86% 

>$1M-$10M 

456 

10 

466 

2.15% 

>$10M-$50M 

115 

3 

118 

2.54% 

>$50M-$1B 

84 

8 

92 

8.70% 

>$1B 

4 

0 

4 

0.00% 

Total 

693 

22 

715 

3.08% 

10.00% 
9.00% 
8.00% 
7.00% 
6.00% 
I  5.00% 
£3  4.00% 

i.  3.00% 
:=  2.00% 
1.00% 
0.00% 


8,70% 


Figure  23.  Failure  Rate  by  Grouped  Dollar  Value 
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Figure  24.  Reasons  for  Grouped  Dollar  Value  Failure 


4.  Level  of  Competition 

The  third  research  question  is  as  follows:  Does  the  level  of  competition  affect 
the  success  of  a  service  contract?  To  answer  this  question,  the  research  grouped 
the  contracts  into  three  categories;  competitive,  non-competitive,  and  other.  The 
researchers  looked  at  these  different  categories  separately  and  examined  the  failure 
rate  of  each  group.  These  categories  are  shown  in  Table  12. 

Finding  3:  Contracts  competed  competitiveiy  had  the  highest  failure 
rate  when  compared  to  the  other  two  forms  of  competition  availabie. 

There  were  540  competitive  contracts  examined  in  the  database.  Of  these 
540  contracts,  17  were  labeled  a  failure,  which  yields  a  failure  rate  of  3.15%.  The 
reasons  that  most  often  resulted  in  a  contract  failure  were  in  the  areas  of 
schedule  and  cost  control,  which  were  each  referenced  seven  times.  The  next 
highest  referenced  source  of  failure  was  management  of  key  personnel,  which 
was  referenced  six  times.  The  failure  rates  due  to  level  of  competition  are  shown 
in  Figure  25.  Non-competitive  contracts  had  the  next  highest  failure  rate  at 
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2.91%.  There  were  172  non-competitive  contracts  in  the  database,  of  which  five 
were  labeled  failures.  Quality  was  referenced  four  times  while  schedule, 
management  of  key  personnel,  and  business  relations  were  each  referenced 
twice.  Contracts  competed  as  other  had  three  contracts  in  the  database  with  zero 
labeled  failures.  The  rating  statistics  across  the  levels  of  competition  are  shown 
in  Figure  26. 


Table  12.  Level  of  Competition  Total  Successes  and  Failures 


Level  of  Competition 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

Basis  (Competitive) 
Basis  (Non- 

523 

17 

540 

3.15% 

Competitive) 

167 

5 

172 

2.91% 

Basis  (Other) 

3 

0 

3 

0.00% 

Total 

693 

22 

715 

3.08% 

3.50% 

3.00% 

2.50% 

lA 

V 

I  2.00% 

I  1.50% 
1.00% 
0.50% 
0.00% 


3.15% 


2.91% 


Basi5  (Competitive) 


Basis  (Non- 
Competitive) 


Basis  of  Contract  .\tvard 


0.00% 

Basis  (Other) 


Figure  25.  Failure  Rates  Among  Level  of  Competition 
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Figure  26.  Levels  of  Competition  Stated  Reasons  for  Failure 


5.  Contract  Type 

The  fourth  research  question  is  as  follows:  Does  the  contract  type  affect 
the  success  of  a  service  contract?  To  answer  this  question,  the  researchers 
grouped  the  contracts  into  six  categories:  CPAF,  combination,  CPFF,  CPIF,  FFP, 
and  other.  The  researchers  looked  at  each  of  these  categories  separately  and 
examined  the  failure  rate  of  each  group,  as  seen  in  Table  13. 

Finding  4:  Contracts  structured  as  a  combination  contract  had  the 
highest  failure  rate  when  compared  to  the  other  five  types  of  available 
contracts. 

There  were  four  combination  contracts  examined  in  the  database.  Of 

these  four  contracts,  two  were  labeled  failures,  which  yields  a  failure  rate  of 

50.0%.  Schedule  and  cost  were  both  referenced  twice  in  the  failed  contracts 

while  quality  and  management  of  key  personnel  were  each  referenced  once.  The 

failure  rates  due  to  contract  type  are  shown  in  Figure  27.  Cost  plus  fixed  fee 

contracts  had  the  next  highest  failure  rate  at  5.56%.  There  were  36  CPFF 

contracts  in  the  database,  of  which  two  were  labeled  failures.  Cost  control  was 
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referenced  twice  and  schedule  was  referenced  once  in  the  failed  contracts. 
Contracts  competed  as  cost  plus  award  fee  had  58  contracts  in  the  database 
with  three  of  them  labeled  as  failures.  This  yielded  a  failure  rate  of  5.17%.  Two  of 
these  failed  contracts  referenced  cost  control  and  business  relations  while  one 
referenced  the  management  of  key  personnel.  Firm  fixed  price  contracts  had  524 
contracts  in  the  database  with  14  of  them  labeled  as  failures.  This  yielded  a 
failure  rate  of  2.68%.  Seven  of  these  failed  contracts  referenced  quality  while  six 
referenced  the  management  of  key  personnel.  Other  contract  types  had  89 
contracts  in  the  database  with  one  labeled  as  a  failure  because  of  quality  and 
schedule,  which  yielded  a  failure  rate  of  1.12%.  There  were  four  cost  plus 
incentive  fee  contracts,  which  had  zero  labeled  failures.  The  rating  statistics 
among  contract  types  are  shown  in  Figure  28. 


Table  13.  Contract  Type  Total  Successes  and  Failures 


Contract  Type 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

Type  (CPAF) 

55 

3 

58 

5.17% 

Type  (Combination) 

2 

2 

4 

50.00% 

Type  (CPFF) 

34 

2 

36 

5.56% 

Type  (CP IF) 

4 

0 

4 

0.00% 

Type  (FFP) 

510 

14 

524 

2.67% 

Type  (Other) 

88 

1 

89 

1.12% 

Total 

693 

22 

715 

3.08% 

66 


Figure  27.  Contract  Type  Failure  Rate 
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Figure  28.  Contract  Type  Stated  Reasons  for  Failure 
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D. 


ADDITIONAL  ANALYSIS 


1.  Analysis  Overview 

The  next  section  shows  the  results  from  the  additional  analysis  the  PPIRS 
database  revealed.  It  includes  the  failure  rates  of  the  MICCs,  duration  of 
contracts,  and  the  MICC  organizational  data.  The  organizational  data  includes 
the  annual  workload,  number  of  actions,  number  of  assigned  1102  billets,  and 
the  percent  of  those  billets  filled. 

2.  MICCs 

The  contracts  examined  came  from  the  five  different  MICC  regions 
described  in  Chapter  III.  The  Fort  Bragg  regional  office  had  55  contracts.  Fort 
Sam  Flouston  had  189  contracts.  Fort  Knox  had  119  contracts.  Fort  Flood  had 
114  contracts,  and  Fort  Eustis  had  238  contracts.  The  MICC  regions  and  their 
associated  data  are  shown  in  Table  12. 

Finding  5:  Contracts  compieted  by  Fort  Flood  had  the  highest  faiiure 
rate  compared  to  the  other  four  MICC  regions. 

There  were  114  contracts  completed  by  Fort  Hood  examined  in  the 
database.  Of  these  114  contracts,  six  were  labeled  failures.  This  yields  a  failure 
rate  of  5.26%.  The  reason  that  most  often  resulted  in  a  contract  being  labeled  a 
failure  was  in  the  area  of  quality,  which  was  referenced  four  times.  Schedule  and 
business  relations  were  each  referenced  three  times.  The  failure  rates  across  the 
MICCs  are  shown  in  Figure  29.  Fort  Knox  had  the  next  highest  failure  rate  at 
5.04%.  There  were  119  contracts  from  Fort  Knox  in  the  database,  of  which  six 
were  labeled  failures.  The  reason  stated  for  these  contracts  was  most  frequently 
in  the  area  of  management  of  key  personnel,  which  was  referenced  three  times. 
Quality,  schedule,  cost  control,  and  business  relations  were  each  referenced 
twice.  Fort  Sam  Houston  had  189  contracts  in  the  database  with  eight  of  them 
being  labeled  failures.  This  yielded  a  failure  rate  of  4.23%.  The  most  commonly 
stated  reason  for  a  contract  being  determined  a  failure  at  Fort  Sam  Houston  was 

in  the  area  of  cost  control  with  five  ratings.  Schedule  was  the  next  most  frequent 
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rating  that  resulted  in  a  failure,  which  was  referenced  four  times.  Fort  Bragg  had 
a  failure  rate  of  1.82%  and  included  55  contracts  (with  one  labeled  failure).  The 
only  rating  that  received  a  failure  label  from  Fort  Bragg  was  in  the  area  of  quality, 
but  this  result  was  driven  from  only  one  contract.  Fort  Eustis  had  the  most 
contracts  in  the  database  at  238  (with  only  one  labeled  failure),  but  also  had  the 
lowest  failure  rate  (0.42%)  of  any  of  the  MICC  contracts  examined.  The  rating 
statistics  across  the  MICCs  are  shown  in  Figure  30. 


Table  14.  MICCs  Total  Successes  and  Failures 


MICC  Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

Fort  Bragg 

54 

1 

55 

1 .82% 

Fort  Sam  Flouston 

181 

8 

189 

4.23% 

Fort  Knox 

113 

6 

119 

5.04% 

Fort  Flood 

108 

6 

114 

5.26% 

Fort  Eustis 

237 

1 

238 

0.42% 

Total 

693 

22 

715 

3.08% 

Houston 


MICCs 


Figure  29.  Failure  Rates  Among  the  MICCs 
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Figure  30.  MICCs  Stated  Reason  for  Failures  Among  the  Ratings 


3.  Duration 

The  duration  of  contracts  span  from  zero  to  1,246  days  and  occurred 
between  2006  and  2013.  The  days  were  broken  into  five  different  categories 
each  containing  90  days.  The  final  category  contains  contracts  lasting  over  365 
days  in  duration.  In  the  first  category,  zero  to  90  days,  there  were  27  contracts. 
There  were  25  contracts  lasting  between  91  and  180  days.  The  181  to  270 
category  contained  52  contracts.  The  271  to  365  day  category  contained  the 
largest  amount  of  contracts,  at  542.  There  were  69  contracts  lasting  over  365 
days.  These  numbers  are  clearly  seen  in  Table  13. 

Finding  6:  Contracts  with  a  duration  between  0-90  days  had  the 
highest  faiiure  rate  compared  to  the  other  four  contract  durations  analyzed. 

There  were  27  0-90  day  contracts  examined  in  the  database.  Of  these  27 
contracts,  two  were  labeled  failures  as  defined  within  the  parameters  of  this 
research,  which  yields  a  failure  rate  of  7.41%.  Quality,  schedule,  and  cost  control 
were  each  referenced  once  in  the  failed  contracts.  The  failure  rates  because  of 
contract  duration  are  shown  in  Figure  31.  271-365  day  contracts  had  the  next 
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highest  failure  rate  at  3.32%.  There  were  542  271-365  day  contracts  in  the 
database,  of  which  18  were  labeled  failures.  Quality,  cost  control,  and  the 
management  of  key  personnel  were  each  referenced  seven  times  while  schedule 
and  business  relations  were  each  referenced  six  times  in  the  failed  contracts. 
181-270  day  contracts  had  52  contracts  in  the  database  with  one  labeled  failure. 
This  yielded  a  failure  rate  of  1.92%.  The  failed  contract  referenced  schedule, 
business  relations,  and  the  management  of  key  personnel.  Contracts  in  excess 
of  365  days  had  69  contracts  in  the  database  with  one  labeled  failure.  This 
yielded  a  failure  rate  of  1.45%.  The  failed  contract  referenced  quality  and 
schedule.  Contracts  between  91-180  days  had  25  contracts  in  the  database  with 
no  labeled  failures.  The  rating  statistics  among  contract  types  are  shown  in 
Figure  32. 


Table  15.  Duration  Total  Successes  and  Failures 


Contract 

Duration 

Categories 

Totai 

Successes 

Totai 

Faiiures 

Total 

Contracts 

Failure 

Rates 

0-90 

25 

2 

27 

7.41% 

91-180 

25 

0 

25 

0.00% 

181-270 

51 

1 

52 

1 .92% 

271-365 

524 

18 

542 

3.32% 

>365 

68 

1 

69 

1 .45% 

Total 

693 

22 

715 

3.08% 

71 


8.00% 

7.00% 

6.00% 

«  5.00% 
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0.00% 


7.41% 
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Figure  31.  Contract  Duration  Failure  Rates 
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Figure  32.  Contract  Duration  Stated  Reasons  for  Failure 
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4.  Contracts  Completed  by  Year 

The  years  were  categorized  based  on  their  completion  year,  and  ranged 
from  2006  to  2013.  There  were  seven  contracts  in  2006,  18  in  2007,  51  in  2008, 
94  in  2009,  165  in  2010,  175  in  2011,  185  in  2012,  and  20  in  2013.  These 
numbers  are  clearly  shown  in  Table  16. 

Finding  7:  2007  had  the  highest  failure  rate  of  all  the  years  analyzed. 

There  was  one  2007  contract  labeled  a  failure,  giving  that  group  a  5.56% 
failure  rate  (see  Figure  33).  In  2007  there  were  18  contracts  listed  in  PPIRS  for 
the  services  analyzed  (see  Table  16).  This  is  a  small  sample,  allowing  the  one 
failure  to  put  it  above  the  average  total  contract  failure  rate  of  3.08%.  The 
contract  that  failed  for  this  year  failed  for  its  business  relations  and  cost  control 
(see  Figure  34).  The  first  year  analyzed  was  2006  and  consisted  of  a  small  data 
group.  There  were  only  seven  contracts  in  the  data  from  this  year.  There  were 
not  any  contracts  labeled  as  failures,  giving  that  group  a  0%  failure  rate  (see 
Figure  33).  There  were  51  contracts  listed  in  PPIRS  for  the  2008-year  group. 
There  were  two  labeled  failures  for  this  year,  giving  2008  a  failure  rate  of  3.92% 
(see  Figure  33).  This  rate  is  higher  than  the  total  contract  average  of  3.08%.  The 
two  failures  were  given  three  reasons  for  their  failures  (see  Figure  34):  quality, 
cost  control,  and  business  relations.  In  2009  there  were  94  contracts  entered  into 
PPIRS.  There  were  five  labeled  failures,  giving  2009  a  5.32%  failure  rate  (see 
Figure  33).  The  2009  group’s  failure  rate  was  higher  than  the  total  contract  failure 
rate  of  3.08%.  The  failing  contracts  for  this  year  listed  four  reasons  for  failing. 
The  most  common  reason  for  failure  of  these  contracts  was  quality  (see  Figure 
34).  Three  other  reasons  were  listed  and  they  were  business  relations, 
management  of  key  personnel,  and  utilization  of  small  business.  During  2010, 
the  number  of  contracts  entered  into  PPIRS  grew  to  165  contracts  (see  Table 
14).  There  were  four  contracts  labeled  as  failures,  giving  the  year  a  2.42%  failure 
rate  (see  Figure  33).  This  year  was  under  the  total  contract  average  failure  rate 
of  3.08%.  The  most  common  reasons  for  failure  for  2010  were  quality  and 
scheduling  (see  Figure  34).  While  not  the  most  common  reasons  for  failing,  there 
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were  three  other  supporting  reasons  contracts  failed  during  this  year.  They  were 
cost  control,  business  relations,  and  management  of  key  personnel.  The 
research  showed  175  contracts  inputted  into  PPIRS  in  2011.  During  this  year 
there  were  five  labeled  failures,  giving  2011  a  failure  rate  of  2.86%  (see  Figure 
33).  The  year  2011  showed  that  its  failure  rate  was  well  below  the  average  total 
contract  failure  rate  of  3.08%.  The  most  common  reason  for  failure  in  2011  was 
for  scheduling  problems  (see  Figure  34).  This  reason  was  listed  for  all  five  of  the 
failures.  There  were  other  reasons  for  failure,  such  as  quality,  cost  control, 
business  relations,  and  management  of  key  personnel.  In  2012  there  were  a  total 
of  185  contracts,  which  included  four  labeled  failures,  giving  2012  a  failure  rate  of 
2.16%,  which  is  below  the  total  contract  average  of  3.08%  (see  Figure  33).  The 
most  common  reason  for  failure  of  these  contracts  was  for  scheduling  (see 
Figure  34).  Scheduling  failures  were  listed  for  two  of  the  contracts.  The  other 
reasons  for  failure  were  quality,  cost  control,  business  relations,  and 
management  of  key  personnel.  During  2013  there  were  20  contracts  entered  into 
PPIRS.  There  was  one  labeled  a  failure,  giving  2013  a  failure  rate  of  5%  (see 
Figure  33).  This  failure  rate  is  higher  than  the  total  contract  average  of  3.08%. 
The  reason  for  the  failure  in  2013  was  cost  control  (see  Figure  34). 


Table  16.  Ending  Year  Total  Successes  and  Failures 


Ending  Year 
Total  Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

2006 

7 

0 

7 

0.00% 

2007 

17 

1 

18 

5.56% 

2008 

49 

2 

51 

3.92% 

2009 

89 

5 

94 

5.32% 

2010 

161 

4 

165 

2.42% 

2011 

170 

5 

175 

2.86% 

2012 

181 

4 

185 

2.16% 

2013 

19 

1 

20 

5.00% 

Total 

693 

22 

715 

3.08% 
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Figure  33.  Failure  Rate  by  Year 
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Figure  34.  Reasons  for  Failure  by  Year 
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5. 


Annual  Workload  Amount  in  Dollars 


Regional  MICC  office  annual  workload  in  dollars  spent  was  grouped  into 
three  categories.  The  categories  consisted  of  contract  totals  between  $0  and 
$500  million,  contracts  greater  than  $500  million  to  $1  billion,  and  contracts 
greater  than  $1  billion.  The  first  category  consisted  of  344  contracts,  the  second 
category  had  256,  and  the  final  category  contained  115  contracts.  These  figures 
are  shown  in  Table  17. 

Finding  8:  Regionai  MICC  offices  that  had  spent  between  $0  and  $500 
miiiion  in  annual  workload  had  the  highest  failure  rate. 

The  failure  rate  for  the  MICC  regions  between  $0  and  $500  million  in 
annual  workload  had  failure  rate  of  4.36%.  These  rates  are  shown  in  Figure  35. 
The  primary  reason  stated  for  failure  was  schedule  for  the  0-$500  million 
category.  The  second  category  had  256  with  a  failure  rate  of  2.34%. 
Management  of  key  personnel  was  stated  as  the  reason  for  failures  for  category 
two  and  the  final  category  contained  115  contracts  with  only  one  of  them  being 
labeled  a  failure,  giving  the  category  a  0.87%  failure  rate.  The  final  category’s 
only  labeled  failure  listed  cost  control  and  business  relations  as  the  reason  for 
failure.  These  results  are  shown  in  Figure  36. 


Table  17.  Annual  Workload  in  Dollars  Total  Success  and  Failures 


Contract  Dollar 
Amount 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

$0-$500M 

329 

15 

344 

4.36% 

>$500M-$1B 

250 

6 

256 

2.34% 

>$1B 

114 

1 

115 

0.87% 

Total 

693 

22 

715 

3.08% 

76 


5.00% 

4.50% 

4.00% 

^  3.50% 

^  3.00% 
cc 

o  2.50% 

I  2.00% 

1.50% 

1.00% 

0.50% 

0.00% 


4.36% 


0.87% 


S0-$500M  >S500M  SIB  >S1B 


Annual  Contract  Workload  in  Dollars 


Figure  35.  Annual  Workload  in  Dollars  Failure  Rates 
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6.  Number  of  Actions  Completed 

The  number  of  actions  a  MICC  completed  in  a  year  was  separated  into 
two  different  categories.  The  first  category  ranged  from  zero  to  3,500  contracts. 
The  second  category  ranged  from  3,501  to  7,000  contracts.  These  categories 
can  be  seen  in  Table  18. 

Finding  9:  Contracts  managed  by  a  MICC  that  completed  3,501-7,000 
contracts  annually  had  the  highest  failure  rate  when  compared  to  MICCs 
that  completed  3,500  or  fewer  contracts  annually. 

There  were  277  contracts  in  this  category  in  the  database.  Of  these  277 
contracts,  nine  were  labeled  as  failures,  which  yields  a  failure  rate  of  3.25%  (see 
Figure  37).  Quality  and  the  management  of  key  personnel  were  each  referenced 
five  times  in  the  failed  contracts.  The  failure  rates  because  of  number  of  completed 
contracts  annually  are  shown  in  Figure  37.  Contracts  completed  in  a  MICC  that 
completed  3500  or  fewer  contracts  annually  had  the  next  highest  failure  rate  at 
2.91%.  There  were  413  of  these  contracts  in  the  database,  of  which  12  were 
labeled  as  failures.  Schedule  and  cost  control  were  each  referenced  five  times 
while  quality  was  referenced  four  times  in  the  failed  contracts.  There  was  no  data 
available  for  25  contracts,  of  which  one  was  labeled  a  failure.  The  rating  statistics 
among  contract  types  are  shown  in  Figure  38. 


Table  18.  Number  of  Actions  Total  Success  and  Failures 


Number  of 
Actions 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

0-3500 

401 

12 

413 

2.91% 

3501-7000 

268 

9 

277 

3.25% 

Data  not  available 

24 

1 

25 

4.00% 

Total 

693 

22 

715 

3.08% 
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Figure  37.  Failure  Rates  by  Number  of  Completed  Contracts  Annually 
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Figure  38.  Number  of  Completed  Contracts  Annually  Stated  Reasons  for  Failure 


7.  Authorized  1102  Billets 

There  were  147  contracts  that  were  completed  by  MICCs  that  had 
between  zero  and  50  authorized  1102  billets,  543  contracts  that  had  over  50 
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authorized  1102  billets,  and  25  contracts  with  no  data  available.  These  figures 
are  shown  in  Table  19. 

Finding  10:  The  category  with  0-50  1102  billets  had  the  highest 
failure  rate. 

This  category  had  147  total  contracts  with  eight  failures,  giving  it  a  failure 
rate  of  5.44%  (see  Figure  39).  The  common  reason  for  these  failures  was  quality. 
This  reason  was  listed  for  five  of  the  eight-labeled  failed  contracts.  The  next 
category  that  contained  51-105  1102  authorized  billets  had  543  total  contracts. 
This  category  had  13  failures,  giving  this  group  a  2.39%  failure  rate.  The  most 
common  reason  for  failure  listed  for  this  group  was  schedule.  This  reason  was 
listed  seven  times  out  of  the  13  labeled  failed  contracts.  The  final  group  did  not 
have  data  on  authorized  billets.  This  group  contained  25  total  contracts  with  one 
failure,  giving  this  category  a  4%  failure  rate. 


Table  19.  Authorized  1102  Billets  Total  Success  and  Failures 


Authorized  1102 
Billets  Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

0-50 

139 

8 

147 

5.44% 

51-105 

530 

13 

543 

2.39% 

Data  not  available 

24 

1 

25 

4.00% 

Total 

693 

22 

715 

3.08% 
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Figure  39.  Failure  Rates  ofMICCs  by  1102  Billet  Authorizations 


Figure  40.  Reasons  for  Failures  by  Billet  Authorization 
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8.  Percentage  of  1102  Billets  Filled 

To  analyze  how  1102  vacancies  affect  contract  success,  the  PPIRS 
contract  data  was  broken  into  six  groups.  These  groups  consisted  of  MICC 
regions  that  had  billets  50-60%  filled,  61-70%  filled,  71-80%  filled,  81-89% 
filled,  90-100%  filled,  and  there  was  contract  data  that  did  not  contain  adequate 
billet  information. 

Finding  11:  The  group  of  1102  billets  that  were  60-70%  filled  had  the 
highest  failure  rate  among  this  variable’s  categories. 

The  60-70%  group  had  a  4.94%  failure  rate  (see  Table  20  and  Figure  41). 
The  group  of  1102  billets  that  were  61-70%  filled  had  81  total  contracts.  The 
most  common  reasons  listed  for  this  group  to  fail  were  schedule  and  cost  control 
(see  Figure  42).  The  group  of  1102  billets  that  were  50-60%  filled  contained  data 
for  22  total  contracts.  This  group  had  one  labeled  a  failure,  which  gave  this  group 
a  4.55%  failure  rate.  This  rate  is  higher  than  the  total  contract  average  of  3.08%. 
The  reason  for  the  failure  in  this  group  was  quality,  scheduling,  and  management 
of  key  personnel.  The  next  group  of  1102  billets  that  were  71%-80%  filled 
consisted  of  122  total  contracts.  There  were  five  contracts  labeled  as  failures, 
which  gave  this  group  a  4.1%  failure  rate.  This  group  was  higher  than  the  total 
contract  failure  rate  of  3.08%.  The  most  common  reasons  listed  for  the  failure  of 
the  contracts  were  schedule  and  cost  control.  The  group  that  contained  billets 
filled  81%-89%  had  233  contracts  with  five  labeled  a  failure.  The  five  failures 
gave  this  group  a  2.15%  failure  rate.  This  group  fell  below  the  total  contract 
failure  rate  of  3.08%.  The  most  common  reasons  for  failure  of  these  contracts 
were  quality,  schedule,  and  business  relations.  The  data  for  the  final  group  that 
had  billets  90%-100%  filled  consisted  of  99  total  contracts.  This  group  contained 
one  labeled  a  failure,  giving  this  group  a  failure  rate  of  1.01%.  The  group  failure 
rate  was  well  below  the  total  contracting  failure  rate  of  3.08%.  The  reason  for  the 
failure  in  this  group  was  management  of  key  personnel. 
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Table  20.  Percentage  of  1102  Billets  Filled  Total  Success  and  Failures 


Percentage  of 
1102  Billet  Filled 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

50-60% 

21 

1 

22 

4.55% 

61-70% 

77 

4 

81 

4.94% 

71-80% 

117 

5 

122 

4.10% 

81-89% 

228 

5 

233 

2.15% 

90-100% 

98 

1 

99 

1.01% 

Data  not  available 

152 

6 

158 

0.00% 

Total 

693 

22 

715 

3.08% 

6.00% 


5.00%  4.55% 


I  4.00% 
^  3.00% 
tS  2.00% 


1.00% 


4.94% 


0.00% 


4.10% 


2.15% 


1.01% 
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Percent  of  Authorized  1102  Billets  Filled 


Figure  41 .  Failure  Rate  for  Different  1102  Billet  Vacancy  Groups 
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Figure  42.  Reasons  for  Failures  of  Different  1102  Vacancy  Groups 


E.  SIGNIFICANCE  TESTING 

The  research  results  were  analyzed  further  by  seeing  if  the  actual  failure 
rates  are  significantly  different  than  what  would  be  expected  if  the  total  contract 
failure  rate  was  applied  to  each  variable.  The  Chi  Square  test  (Fisher’s  exact 
test)  was  used  to  test  the  different  variables.  The  null  hypothesis  for  this  test  is 
that  the  category  failure  rates  within  the  variables  are  not  significantly  different 
than  the  total  contract  failure  rate  (3.08%).  The  null  hypothesis  is  rejected  if  the 
p-value  for  the  variable  is  less  than  0.05. 

This  test  showed  only  three  variables  that  had  categories  that  were 
significantly  different  when  the  average  total  contract  failure  rate  was  applied  to 
the  variable  and  category  (3.08%).  These  variables  were  contract  amount  (p= 
.036),  contract  type  (p  =.009),  and  contracts  completed  by  MICC  region  (p=.011). 
The  biggest  reason  for  a  significant  finding  for  contact  amount  was  the  category 
of  $50  million-  $1  billion.  If  the  total  contract  failure  rate  were  applied  to  this 
category,  it  would  have  expected  to  fail  one  time,  but  in  actuality  it  had  eight 
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failures.  Within  the  contract  type  variable,  three  contract  types  had  a  higher 
difference  than  what  would  have  been  expected.  Combination  type  contracts 
(contracts  that  used  a  combination  of  CPAF,  CPFF,  and  FFP  together)  was 
expected  to  have  0.1  failures,  but  actually  had  two.  FFP  type  contracts  were 
expected  to  have  16,  but  had  14.  Other  t/pe  contracts  (such  as  labor  hours  and 
time  and  materials)  was  expected  to  have  three  failures,  but  only  had  one.  The 
final  variable  that  showed  a  significant  difference  was  contracts  by  MICC  region. 
There  were  three  reasons  that  showed  a  significant  difference  than  what  was 
expected  and  the  actual  failures.  Ft.  Eustis  was  expected  to  have  7.3  failures,  but 
only  had  one.  Ft.  Knox  and  Ft.  Flood  categories  were  both  expected  to  have  3.5 
failures,  but  had  six  failures  each. 


Table  21 .  Chi  Square  and  Fisher’s  Exact  P-Value  Test  Results 


Contract  Variables 

p-value 

Significant? 

Type  of  Service  (RJSD) 

0.761 

No 

Contractual  Amounts 

0.036 

Yes 

Level  of  Competition 

1.00 

No 

Contract  Type 

0.009 

Yes 

MICC 

0.011 

Yes 

Duration 

0.567 

No 

End  Year 

0.619 

No 

Annual  Workload  in  Dollars 

0.142 

No 

Number  of  Completed  Actions 

0.709 

No 

Number  of  Authorized  1102  Billets 

0.098 

No 

Percent  of  1 1 02  billets  Filled 

0.301 

No 

The  remainder  of  the  variables  showed  a  p-value  greater  than  0.05  (see 
Table  21).  This  means  the  null  hypothesis  is  not  rejected.  While  the  results  of  the 
different  categories’  failure  rates  fluctuated,  they  were  not  significantly  different 
from  the  total  contract  failure  rate.  These  fluctuations  could  be  from  acceptable 
randomness  within  the  data.  The  following  section  discusses  the  results  and 
managerial  implications  from  the  research  analysis  and  potential  areas  the  DoD 
can  focus  on  to  reduce  the  failure  rates  of  specific  service  contracts. 
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F.  DISCUSSION  OF  RESULTS  AND  MANAGERIAL  IMPLICATIONS 

1.  Discussion  Overview 

The  purpose  of  this  research  is  to  identify  variables  in  the  service 
contracting  process  that  promote  successful  service  contracts.  This  section 
examines  possible  reasons  for  the  results  found  in  the  research.  It  also  considers 
each  variable  the  researchers  found  extraordinary  and  the  categories  of  these 
variables.  Failed  contracts  are  examined  by  looking  at  the  Service  Acquisition 
Process  covered  in  Chapter  II,  and  this  section  discusses  where  these  failures 
could  be  addressed  to  assist  government  contracting  agencies.  In  discussing  the 
implications  of  the  research,  it  had  to  be  assumed  that  the  contract  was  proper 
(based  on  fair  and  reasonable  price  and  accurately  reflecting  schedule  and 
performance  requirements)  in  terms  of  the  requiring  agencies’  needs.  This 
allowed  proper  contract  administration  to  discover  deficiencies  with  the 
contractor’s  work  and  proper  documentation  into  CPARS. 

•  Finding  1:  The  S  type  contracts  had  the  highest  failure  rate  of 
all  the  product  service  codes  analyzed. 

The  first  variable  discussed  is  how  the  different  service  types  affected  the 
success  of  a  contract.  It  was  found  that  the  types  of  services  were  not 
significantly  different  than  the  overall  total  contract  failure  rate.  Even  though 
contract  types  are  not  shown  to  be  statistically  significant  in  terms  of  affecting 
contract  success,  it  was  observed  that  utilities  and  housekeeping  (S)  failed  six 
times  for  business  relations  and  six  times  for  management  of  key  personnel  out 
of  the  11  reported  failures.  The  business  relation’s  assessment  contains  the 
assessment  of  the  contractors’  ability  to  coordinate  their  business  activities,  such 
as  their  attitude  towards  customers,  customer  satisfaction,  and  their  cooperation 
(DoD,  2011).  The  business  relation’s  category  could  be  more  closely  addressed 
during  the  Planning  Phase  of  the  service  acquisition  process.  Step  three  of  this 
process  is  Market  Research.  One  possible  way  of  mitigating  future  failures  of  this 
type  is  to  ensure  that  the  contracting  agency  gives  this  step  extra  attention.  This 
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step  contains  seven  sub-elements  that  ensure  the  acquisition  team  has  a 
detailed  understanding  of  current  market  practices  in  the  area  of  the  service 
being  provided.  This  will  assist  with  making  performance  requirements  more 
attainable  for  the  contractor  (DoD,  2012).  The  next  common  failure  for  utilities 
and  housekeeping  services  was  the  management  of  key  personnel.  This 
assessment  area  measures  the  contractor’s  ability  to  maintain  qualified 
individuals  in  key  positions  as  outlined  in  the  contract  (DoD,  2011).  These 
numbers  suggest  that  the  contracting  agency  should  focus  on  the  Execution 
Phase  of  the  process  and  give  extra  attention  to  the  Execute  Strategy  step.  This 
phase  and  step  goes  from  the  issuance  of  the  RFP  to  the  Post  Award 
implementation/transition.  Within  this  step  is  the  source  selection.  Prior  to 
conducting  the  source  selection  process,  the  acquisition  team  needs  to  ensure 
that  the  source  selection  evaluators  (SSE)  are  properly  trained.  The  team  must 
also  ensure  that  the  SSEs  understand  how  to  properly  evaluate  the  offeror’s 
management  proposal.  This  proposal  gives  the  offeror’s  breakdown  of  key 
personnel  that  will  be  involved  in  the  contract. 

•  Finding  2:  The  contract  grouping  that  consisted  of  contracts 
worth  a  doiiar  amount  greater  than  $50  miilion-  $1  billion  had 
the  highest  failure  rate  of  all  the  groups  in  the  analysis. 

As  discussed  in  the  previous  section,  this  variable  proved  to  have  a 
significant  impact  on  service  contracting  success.  The  category  within  this 
variable  appearing  to  be  the  most  significant  was  the  category  containing 
contracts  greater  than  $50  million-$1  billion.  This  group  had  the  highest  failure 
rate  at  8.7%.  The  most  common  reason  for  this  failure  was  cost  control.  The  cost 
control  rating  contains  an  assessment  of  the  contractors’  ability  to  “forecast, 
manage,  and  control  the  cost”  associated  with  conducting  their  services  (DoD, 
2011,  p.  A3-7).  Again,  this  research  assumes  that  the  contractor  was  provided  a 
proper  contract,  which  in  turn  allowed  proper  contract  administration  that  resulted 
in  an  identified  cost  control  failure  documented  in  CPARS.  This  leads  one  to 
assume  that  a  contract  was  awarded  to  a  business  that  was  not  able  to  perform 


87 


the  cost  control  function  sufficiently.  This  leads  back  to  the  Execute  Phase  and 
Execute  Strategy  Step  needing  extra  attention  to  mitigate  future  failures.  The 
acquisition  team  needs  to  ensure  that  all  SSEs  are  properly  trained  on  how  to 
accurately  evaluate  the  cost  proposals  to  ensure  the  business  fully  understands 
the  contract  requirements.  If  the  cost  proposal  is  not  properly  evaluated,  a 
contractor  may  be  awarded  the  contract  that  does  not  fully  grasp  the  contract 
requirements  leading  to  future  cost  overruns.  Another  category  within  this  data 
was  worth  analyzing:  The  group  that  contained  contract  dollar  amounts  greater 
than  $1  million-$10  million.  While  this  category’s  failure  rate  was  actually  one  of 
the  lowest  failure  rates  (2.15%),  the  reason  for  its  failure  appears  to  be  important 
for  analysis.  This  category  had  a  total  of  10  failures.  Of  these  10  failures,  seven 
were  listed  for  quality  reasons.  The  quality  assessment  factor  contains  an 
evaluation  of  the  contractor’s  qualitative  performance  and  compares  it  to  the 
requirements  stated  in  the  contract  (DoD,  2011).  Since  these  contract  failures 
were  proper  contracts  allowing  proper  documentation  in  to  CPARS,  the 
acquisition  team  needs  to  ensure  they  focus  on  training  of  their  SSEs.  Acquisition 
teams  that  are  a  part  of  contracts  within  this  dollar  amount  need  to  focus  the 
Execute  Phase  and  Execute  Strategy  step  of  the  Services  Acquisition  Process. 
They  need  to  ensure  their  SSEs  are  properly  trained  and  can  evaluate  the 
contractor’s  technical  proposal.  This  proposal  will  help  the  acquisition  team’s 
understanding  of  the  contractor’s  plan  to  complete  the  quality  requirements  in  the 
contract. 


•  Finding  3:  Contracts  competed  competitively  had  the  highest 
failure  rate  when  compared  to  the  other  two  forms  of 
competition  available. 

Competitively  competed  contracts  had  the  highest  of  the  category’s  failure 
rates  at  3.15%.  These  contracts  most  often  failed  because  of  quality  and 
schedule.  While  out  of  523  contracts  only  17  failed,  this  gave  the  category  a 
slightly  higher  failure  rate  than  the  total  contract  failure  rate  of  3.08%.  This 
category  failed  seven  times  for  scheduling  reasons  and  seven  times  for  cost 
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control.  The  numbers  in  these  two  areas  suggest  the  Execute  Phase  and  the 
Execute  Strategy  step  in  the  Service  Acquisition  Process  needs  to  be  scrutinized. 
The  acquisition  team  needs  to  ensure  proper  training  of  the  SSEs  on  how  to 
properly  evaluate  the  contractor’s  cost  proposal  and  technical  proposal.  These 
proposals  show  sufficient  understanding  to  successfully  fulfill  the  contract’s 
requirements.  If  the  cost  proposal  is  not  adequately  evaluated,  this  may  result  in 
future  cost  overruns.  Inadequate  evaluation  of  the  contractor’s  technical  proposal 
could  result  in  the  contractor’s  inability  to  meet  scheduling  requirements. 

•  Finding  4:  Contracts  structured  as  a  combination  contract  had 
the  highest  failure  rate  when  compared  to  the  other  five  types 
of  available  contracts. 

The  contract  type  variable  was  significant  on  the  outcome  of  the  service 
contract.  There  are  three  categories  worth  mentioning  in  this  variable.  They  are 
the  combination,  CPAF,  and  CPIF.  The  combination  contract  type  showed  a 
failure  rate  of  50%.  This  could  be  because  of  the  relatively  low  number  of  this 
contract  type  in  the  data.  The  data  consisted  of  only  four  contracts  and  two  of 
these  showed  failures.  The  other  two  categories  (CPAF-5.17%,  CPFF-  5.56%) 
showed  a  failure  rate  nearly  double  that  of  the  next  lowest  category  (FFP, 
2.67%).  Contracting  officers  using  these  types  of  contract  vehicles  for  their 
procurement  of  services  should  be  aware  of  these  failure  rates.  The  leading  and 
reoccurring  theme  among  all  three  of  these  categories  was  cost  control.  The  cost 
type  contracts  places  more  financial  responsibility  on  the  government.  These 
contract  vehicles  should  be  avoided  when  the  requirements  of  the  service  allow 
for  a  FFP  contract.  To  gain  better  knowledge  and  build  better  requirements  to 
allow  for  a  FFP  contract.  Step  2:  Review  Current  Strategy  of  the  Planning  Phase 
could  help  mitigate  the  use  of  cost-based  contracts.  This  step  involves  analyzing 
requirements  and  desired  results  of  current  ongoing  service  contracts  and 
attempts  to  draw  parallels  to  gather  necessary  information  to  develop  exact 
requirements  needed  by  the  service.  Thorough  execution  of  Step  3:  Market 
Research  in  the  Planning  Phase  could  also  assist  with  moving  away  from  the  use 
of  these  cost-based  contracts  and  place  more  cost  responsibility  on  the 
contractor.  Market  research  entails  gaining  a  detailed  understanding  of  what  the 
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marketplace  associated  with  the  service  can  provide.  This  understanding  assists 
the  acquisition  team  and  avoids  wasting  taxpayer  dollars  (DoD,  2012).  The 
acquisition  team  needs  to  examine  the  Execute  Strategy  in  the  Execute  Phase 
and  ensure  proper  training  of  its  SSEs.  The  cost  proposals  of  these  cost 
contracts  may  not  have  been  evaluated  correctly.  Without  proper  evaluation  a 
contractor  without  adequate  understanding  of  the  requirements  may  be  awarded 
the  contract.  This  will  lead  to  a  greater  possibility  of  cost  overruns. 

•  Finding  5:  Contracts  completed  by  Fort  Hood  had  the  highest 
failure  rate  compared  to  the  other  four  MICC  regions. 

The  MICC  region  variable  was  examined  and  was  shown  to  have  a 
significant  effect  on  the  success  of  service  contracts.  There  were  three  reasons 
that  showed  a  significant  difference  in  the  expected  category  failures  and  the 
actual  failures.  Ft.  Hood  and  Ft.  Knox  regional  categories  both  were  expected  to 
have  3.5  failures  but  actually  had  six  failures.  Ft.  Hood  had  the  highest  failure 
rates,  and  at  first  glance,  this  finding  may  appear  to  be  negative.  This  finding  may 
have  resulted  from  the  proper  use  and  documentation  within  the  CPARS  system, 
thus  showing  a  more  realistic  outcome  of  contracting  results.  This  MICC  region 
may  have  more  CORs  assigned  to  contracts  under  their  control  allowing  for 
better  oversight  and  assessing  contract  failure  when  appropriate.  The  most  cited 
reason  for  failure  in  this  region  was  for  quality.  Acquisition  teams  within  this 
region  may  want  to  examine  how  the  technical  proposals  are  being  evaluated  in 
the  Execution  Strategy  step.  They  need  to  ensure  their  SSEs  are  properly  trained 
to  mitigate  future  failures  for  quality  reasons.  Ft.  Knox  did  not  have  a  pattern  for 
reasons  of  failure.  They  were  fairly  evenly  distributed  and  did  not  show  a  possible 
weakness  in  the  process  to  be  improved.  The  next  MICC  region  that  appeared 
significant  was  Ft.  Eustis.  The  Ft.  Eustis  region  fell  significantly  below  the 
expected  failure  rate.  This  finding  also  may  be  because  of  manning  shortages  or 
training  deficiencies.  This  region  may  not  have  the  CORs  to  enforce  proper 
contract  administration  and  show  deficient  past  performance  records  through 
CPARS. 
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•  Finding  6:  Contracts  with  a  duration  between  0-90  days  had 
the  highest  failure  rate  compared  to  the  other  four  contact 
durations  analyzed. 

As  discussed  in  the  previous  section,  contract  duration  was  found  not  to 
have  any  significantly  different  failure  rates  than  the  total  contract  failure  rate. 
The  first  category  of  contracts  with  a  duration  of  0-90  days  had  a  failure  rate  of 
7.41%.  The  small  sample  size  explains  this  high  failure  rate.  The  271-365  day 
contractual  duration  had  failures  for  all  the  reasons  listed  that  was  almost  evenly 
distributed.  The  data  did  not  show  that  duration  was  meaningful  to  the  success  of 
a  service  contract. 

•  Finding  7:  2007  had  the  highest  failure  rate  of  all  the  years 
analyzed. 

After  the  significance  test  was  completed  on  this  variable  it  showed  that  it 
was  not  significantly  different  than  the  total  contract  failure  rate.  While  not 
significant,  the  year  2007  had  18  contracts  with  one  failure,  giving  it  the  highest 
failure  rate  at  5.56%.  It  failed  one  time  for  cost  control  and  business  relations. 
This  high  failure  rate  appears  to  be  the  result  of  a  small  sample  size.  The  year 
2011  had  a  lower  failure  rate  than  most  of  the  other  years  included  in  the  data. 
The  data  showed  2011  had  five  failed  contracts.  All  of  these  failures  also  had 
schedule  listed  as  a  failed  evaluation  criteria.  This  criterion  is  an  assessment  of 
the  ability  of  the  contractor  to  meet  schedules  outlined  in  the  contract  such  as 
“task  orders,  milestones,  schedules,  and  administrative  requirements”  (DoD, 
2012,  p.  A3-7).  These  failures  may  have  been  mitigated  through  a  more  thorough 
contracting  process  such  as  conducting  a  more  deliberate  execution  phase.  This 
phase  consist  of  the  pre-solicitation  conference  giving  the  contractor  and  agency 
requiring  the  service  a  more  common  understanding  of  expectations  and 
requirements  (Apte  et  al.,  2006).  These  findings  may  also  show  that  in  2011  the 
planning  phase  and  development  phase  was  conducted  more  thoroughly  than  in 
previous  years,  allowing  the  agencies  to  better  conduct  contract  administration 
and  hold  the  contractors  responsible  for  scheduling  deficiencies  and 
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documenting  these  deficiencies  in  CPARS.  The  2012  and  2013  year  groups 
show  a  decline  in  scheduling  failures,  allowing  one  to  assume  agencies  and 
contractors  were  able  to  come  to  a  better  common  understanding  of  scheduling 
requirements. 


•  Finding  8:  Regionai  MICC  offices  that  had  spent  between  0  and 
$500  miilion  in  annuai  workioad  had  the  highest  faiiu re  rate. 

Regional  MICC  offices  that  did  between  0-$500  million  in  annual  workload 
had  the  highest  failure  rate  at  4.36%.  Its  most  common  reasons  for  failure  were 
schedule  and  quality.  To  mitigate  future  failures  in  the  same  areas,  acquisition 
teams  should  focus  on  the  Execution  Phase  and  Execution  Strategy  step  within 
the  Service  Acquisition  Process.  This  step  is  the  source  selection  for  the 
contract.  The  acquisition  team  needs  to  ensure  proper  training  of  its  SSEs  to 
ensure  a  good  evaluation  of  the  technical  proposal  within  the  contractor’s  offer. 
Without  proper  training  on  how  to  evaluate  these  proposals,  the  government  is  at 
risk  of  awarding  a  contract  to  a  contractor  that  is  unable  to  comply  with  the 
contract  scheduling  and  quality  requirements.  While  the  different  annual 
workload  failure  rates  did  not  show  that  they  were  statistically  different  than  the 
total  contract  failure  rate,  the  numbers  did  appear  to  have  an  overarching  pattern. 
As  the  annual  workload  increased,  the  failure  rate  decreased.  The  Chi  Square 
test  shows  that  these  differences  could  be  because  of  randomness,  and  how  the 
data  was  gathered  for  analysis,  there  does  appear  to  be  a  trend  within  this  data. 
These  findings  appear  to  be  counterintuitive  to  what  one  might  expect.  This  trend 
could  be  explained  through  how  the  MICCs  were  staffed.  These  MICC  offices 
may  have  been  projected  to  have  greater  dollar  amounts  passing  through  their 
regions  because  of  deployments  and  higher  operational  tempos  in  their  areas. 
Given  this  projection,  the  MICC  offices  may  have  been  prioritized  and  manned 
with  more  personnel.  This  would  allow  for  more  people  to  conduct  the  service 
acquisition  process  more  thoroughly  and  with  greater  success.  This  greater 
workload  would  also  allow  for  a  higher  learning  curve  and  allow  the  acquisition 
teams  within  these  regions  to  better  understand  their  jobs  and  responsibilities.  A 
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counterargument  to  this  could  be  that  these  higher  workloads  do  not  allow  for  the 
execution  phase  to  be  properly  conducted.  This  allows  for  less  than  proper 
contract  administration  and  failed  contracts  not  properly  documented  on  CPARS. 

•  Finding  9:  Contracts  managed  by  a  MICC  that  completed 
3,501-7,000  contracts  annually  had  the  highest  failure  rate 
when  compared  to  MICC’s  that  completed  3,500  or  fewer 
contracts  annually. 

The  MICC  regions  that  consisted  of  contracting  actions  between  3,501- 
7,000  contracts  annually  had  the  highest  failure  rates.  This  category  failed  most 
commonly  for  quality  and  management  of  key  personnel.  These  failures  in 
CPARS  suggest  the  MICCs  that  fall  within  these  categories  need  to  look  at  how 
they  are  executing  their  source  selection  within  the  execution  strategy  step.  The 
MICCs  need  to  ensure  the  acquisition  teams  are  properly  training  their  SSEs  on 
how  to  evaluate  the  contractor’s  management  and  technical  proposals. 
Acceptance  of  a  contractor’s  proposal  without  proper  understanding  of  the 
contract  requirements  raises  the  risk  of  contract  failure.  As  discussed  previously, 
this  variable  had  p-value  greater  than  0.05,  showing  that  its  categories’  failure 
rates  were  not  significantly  different  than  what  would  have  been  expected  given 
the  total  contract  failure  rate.  While  not  significantly  different,  the  numbers  do 
appear  to  reflect  a  trend  that  as  the  workload  increases,  the  failure  rate  also 
increases.  While  the  Chi  Square  test  shows  that  these  differences  may  be  from 
randomness  within  the  data  population,  this  trend  does  allow  the  researchers  to 
speculate.  It  appears  that  as  the  contracting  workload  increases,  the  number  of 
trained  personnel  for  government  contracting  is  not  being  adequately  increased. 
GAO  (2013b)  and  Gansler  (2011)  have  both  identified  that  without  the  proper 
trained  personnel  to  conduct  the  contracting  process,  the  government  runs  the 
risk  of  contract  failure.  As  discussed  in  Chapter  I,  history  has  shown  dramatic 
increase  in  service  contracting  over  the  past  21  years  (Ellman  et  al.,  2011).  With 
this  increase,  it  is  important  to  increase  the  proper  workforce  as  well. 
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•  Finding  10:  The  category  with  0-50  1102  biiiets  had  the  highest 
faiiure  rate. 

Mice  regions  that  had  0-50  1102  billets  had  the  highest  failure  rate  at 
5.44%.  Its  most  common  reason  for  failing  was  quality.  These  numbers  may 
suggest  the  acquisition  team  within  these  categories  need  to  focus  their  attention 
on  the  Execution  Phase.  Within  this  phase  the  teams  need  to  examine  how  they 
are  training  their  SSEs.  These  numbers  may  suggest  they  are  not  awarding  the 
contract  to  the  best  contractor  or  their  requirements  for  this  area  are  not 
attainable  by  the  contractor.  While  this  variable  was  shown  not  to  be  statistically 
significant,  the  results  of  the  research  did  appear  to  have  an  obvious  trend.  This 
data  showed  as  the  authorized  1102  billets  increased,  the  failure  rate  decreased. 
While  the  Chi  Square  test  p-value  states  these  results  may  be  from  randomness, 
it  appears  to  be  a  result  one  would  expect  to  find.  One  can  speculate  that  MICC 
offices  with  higher  authorized  1102  billets  also  have  more  1102  specialists  in  the 
position,  whether  all  the  positions  are  filled  or  not.  The  results  of  this  research 
support  the  GAO  (2013b)  and  report  Gansler  (2011)  published  stating,  without 
properly  trained  personnel  to  conduct  the  contract  process  the  government  runs 
the  risk  of  contract  failure. 

•  Finding  11:  The  group  of  1102  biiiets  that  were  60-70%  fiiled 
had  the  highest  faiiure  rate  among  this  variable’s  categories. 

The  1102  billet  percentage  category  that  showed  60-70%  filled  had  the 
highest  failure  rate  at  4.94%.  It  showed  to  fail  most  often  because  of  schedule 
and  cost  control.  These  numbers  suggest  that  MICCs  within  these  fill  rates  need 
to  focus  on  the  execution  strategy  step  to  mitigate  their  risk  of  future  failures  in 
the  same  areas.  The  MICCs  need  to  look  at  how  they  are  training  their  SSEs  in 
the  areas  of  evaluating  the  cost  and  technical  proposals.  These  two  proposals 
lay  out  how  the  contractor  plans  to  fulfill  the  contracts  requirements  in  forecasting 
cost  and  meeting  scheduling  requirements.  These  proposals  show  if  the 
contractor  really  understands  the  contract  requirements.  If  the  contract  is 
awarded  to  a  contractor  that  does  not  understand  the  contract  requirements  the 
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government  is  at  higher  risk  for  cost  overruns  and  schedule  delays.  This 
variable’s  failure  rates  again  proved  to  be  not  significantly  different  than  the  total 
contract  failure  rate.  The  Chi  Square  test  showed  that  these  differences  in  the 
variables  failure  rates  could  be  due  to  randomness  or  how  the  data  was 
gathered.  The  results  of  this  analysis  had  a  straightforward  trend  that  is  hard  to 
ignore.  The  trend  in  this  analysis  showed  as  the  percentage  for  billets  filled 
increased,  the  failure  rate  decreased.  This  trend  again  supports  the  GAO  (2013b) 
and  Gansler  (2011)  claim  of  the  importance  of  having  trained  personnel  in  the 
positions  for  proper  contracting  process  phases.  Another  trend  identified  in  this 
variable  was  that  the  cost  control  failures  decreased  as  the  billets  are  filled  to 
more  appropriate  levels.  The  direction  of  these  numbers  show,  as  there  are  less 
trained  personnel,  the  government  does  run  the  risk  of  paying  too  much  and 
higher  risk  of  contract  failure. 

G.  SUMMARY 

This  chapter  began  by  reviewing  an  overview  of  the  data.  This  chapter 
then  analyzed  the  data  to  answer  the  research  questions.  The  chapter  covered 
how  the  research  used  the  Chi  Square  test  on  the  data.  This  showed  if  the 
average  total  contract  failure  rate  was  applied  across  the  categories  of  the 
variables  whether  there  would  be  a  significant  difference  from  what  was  actually 
captured  within  the  data.  The  final  section  in  this  chapter  was  a  discussion  of 
results  and  managerial  implications.  This  discussed  in  further  detail  what  was 
found  in  the  analysis  of  the  variables’  failure  rates  and  categories’  reasons  for 
failures.  Chapter  V  covers  the  summary  of  the  research,  conclusion  of  the 
findings,  and  areas  for  further  research  to  enhance  the  knowledge  of  service 
contracting. 
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V.  SUMMARY,  CONCLUSION,  AND  AREAS  FOR  FURTHER 

RESEARCH 


A.  SUMMARY 

Over  the  last  few  decades,  services  contracting  has  continued  to  grow. 
Contracting  for  services  has  grown  in  relation  to  product  contracting  over  the  last 
21  years,  and  was  the  fastest  growing  area  for  DoD  contract  spending  at  a 
growth  rate  of  6.1%  (Ellman  et  al.,  2011).  This  growth  in  dollars  spent  has 
brought  increased  political  attention  and  scrutiny. 

Contract  management  has  been  listed  on  the  GAO’s  High  Risk  Series 
since  1992  (GAO,  2013a).  Problems  in  contract  administration  have  also  resulted 
in  142  reports  completed  by  the  DoD  IG  in  five  years  (Burton,  2009).  Gansler 
(2011)  identified  that  most  current  regulations,  policies,  training,  standards, 
education,  and  management  structures  re  set  up  to  optimize  product  acquisition, 
and  only  a  few  focused  on  the  optimization  of  service  acquisition. 

The  DoD  has  responded  to  these  problems  to  improve  service  acquisition 
in  several  different  ways.  USD  AT&L  Frank  Kendall  released  BBP  2.0  to  place 
senior  leadership  to  manage  service  acquisitions.  The  DoD  also  forced  the 
components  to  adopt  a  standardization  of  taxonomy  in  order  to  better  classify 
acquisitions  (GAO,  2013b). 

With  these  improvements,  service  acquisitions  still  have  problems.  The 
problems  could  be  from  a  lack  of  standard  definition  for  success.  With  the 
differing  goals  and  objectives  of  the  different  stakeholders,  a  proxy  definition  of 
success  is  needed  to  measure  success.  Since  contract  success  and  failure  is 
recorded  through  CPARS,  this  information  is  used  for  the  proxy  definition  for 
success. 
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B.  CONCLUSION 


1.  Research  Findings 

The  purpose  of  this  research  was  to  identify  variables  in  the  service 
contracting  process  that  promote  successful  service  contracts.  To  identify  these 
variables,  the  research  was  focused  on  answering  these  four  research  questions: 

•  Do  the  types  of  services  being  acquired  affect  the  success  of  a 
service  contract? 

•  Do  the  contractual  amounts  affect  the  success  of  a  service 
contract? 

•  Does  the  level  of  competition  used  affect  the  success  of  a  service 
contract? 

•  Does  the  contract  type  affect  the  success  of  a  services  contract? 

In  answering  the  first  research  question,  it  was  discovered  that  the  types 
of  services  being  acquired  is  not  a  significant  influencer  on  the  success  of  a 
service  contract  (p  =  .761).  The  category  of  S-type  services  (utilities  and 
housekeeping)  did  have  the  highest  failure  rate  at  3.77%,  and  is  closely  followed 
by  J-type  services  (maintenance,  repair,  and  rebuilding  of  equipment)  at  3.45%, 
R-type  services  (professional  administrative  and  management  support)  at  2.72%, 
and  D-type  services  (automatic  data  processing  and  telecommunications)  at  0%. 
Though  the  failure  rates  were  different  across  the  service  types  this  research 
examined,  the  overall  failure  rate  was  close  to  the  rate  expected  for  the  types  of 
services  examined  in  this  research. 

In  answering  the  second  research  question,  the  data  indicated  that  a 
statistically  significant  relationship  existed  between  contractual  amounts  and  the 
success  of  a  service  contract  (p=.036).  Contracts  ranging  from  $50  million-$1 
billion  had  a  failure  rate  over  three  times  higher  than  the  lower  dollar  categories 
(8.70%  vs.  2.86%).  It  was  also  found  that  the  single  highest  point  of  failure  in  the 
failing  contracts  was  cost  control  at  six  references  out  of  the  eight  failures.  Cost 
control  was  noted  as  a  source  of  failure  six  times,  which  was  twice  as  often  as 
the  next  most  common  source  of  failure  which  was  schedule  at  three  times. 
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In  answering  the  third  research  question,  the  data  indicated  that  there  was 
not  a  statistically  significant  relationship  between  the  level  of  competition  used 
and  the  success  of  a  service  contract  (p=  1.00).  Competitive  and  non-competitive 
contracts  did  have  failure  rates  at  3.15%  and  2.91%,  respectively.  Though  the 
failure  rates  were  different  between  the  level  of  competition  used,  the  overall 
failure  rate  was  close  to  the  rate  expected  for  the  types  of  services  examined  in 
this  research. 

Lastly,  in  answering  the  fourth  research  question,  the  data  indicated  that 
there  is  a  statistically  significant  relationship  between  the  contract  type  and  the 
success  of  a  service  contract  (p=  .009).  Contracts  that  used  a  combination  of 
CPFF,  FFP,  and  CPAF  had  the  highest  failure  rate  in  the  database  at  50%, 
although  it  must  be  stated  that  the  sample  size  of  combination  contracts  was 
over  130  times  smaller  than  the  largest  population  in  the  same  category  (FFP- 
522  entries).  The  stated  reason  the  contracts  were  labeled  as  a  failure  in  all  of 
the  combination  contracts  was  schedule  and  cost  control.  The  data  also  shows 
that  the  Army  uses  firm  fixed  price  contracts  for  services  over  five  times  more 
often  than  the  next  most  commonly  used  contract  type.  Firm  fixed  price  contracts 
also  have  the  third  lowest  failure  rate,  which  indicates  that  firm  fixed  price 
contracts  should  continue  to  be  used  as  often  as  possible  because  of  their 
historically  low  failure  rate.  The  results  to  the  research  questions  can  also  be 
seen  in  Table  22. 
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Table  22.  Contract  Variables  Effect  on  Success  of  a  Service  Contract 


Contract  Variables 

Effect  on 
success  of  a 
service  contract 

Type  of  Service  (RJSD) 

No 

Contractual  Amounts 

Yes 

Level  of  Competition 

No 

Contract  Type 

Yes 

2.  Additional  Findings 

While  compiling  the  database,  several  additional  variables  were  analyzed 
which  yielded  some  interesting  results  regarding  variables  that  influence  the 
success  of  service  contracts.  These  additional  variables  were  failure  rates  by 
Mice,  duration  of  contracts,  contracts  by  fiscal  year,  annual  contract  workload  in 
dollars,  and  the  percent  fill  of  authorized  1102  billets  by  MICC. 

Of  all  these  additional  variables  analyzed,  only  the  failure  rates  of  the 
contracts  completed  by  MICCs  were  statistically  significant  (p  =  .011).  The  data 
clearly  showed  that  Ft.  Eustis  had  an  overwhelmingly  lower  failure  rate  (0.42%) 
for  service  contracts  than  the  other  four  MICCs  that  were  analyzed.  Ft.  Eustis 
had  a  failure  rate  over  four  times  lower  than  the  next  lowest  failure  rate  (Ft.  Bragg 
was  1.82%)  and  over  12  times  lower  than  the  highest  failure  rate  (Ft.  Hood  was 
5.26%).  Additional  analysis  is  needed  to  determine  what  caused  these  failure 
rates  to  be  different  among  the  MICCs. 

3.  Recommendations 

Even  with  the  low  failure  rate,  the  data  showed  three  statistically 
significant  relationships.  Contractual  amounts,  contract  types,  and  MICCs  all  had 
a  p-value  of  less  than  0.05.  Contracts  valued  at  $50  million  up  to  $1  billion  had 
the  highest  failure  rate  at  8.70%.  Cost  control  was  referenced  six  times  as  the 
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reason  a  contract  was  labeled  as  a  failure,  which  suggests  that  contracts  in  this 
category  need  better  procurement  planning  during  the  contracting  process,  as 
discussed  in  Chapter  IV.  Contract  types  that  used  a  combination  type  contract 
had  the  largest  failure  rate  of  any  of  the  other  contract  types  at  50%.  CPIF  and 
CPAF  also  had  high  failure  rates.  This  suggests  that  these  contracts  needed 
better  procurement  planning  and  solicitation  planning.  These  two  contracting 
phases  is  where  the  requirements  of  the  services  are  examined  and  contract 
vehicles  chosen.  The  SOW  and  project  scope  needs  to  be  developed  and 
matured  enough  to  allow  for  FFP  contract  to  be  accepted  by  the  contractor  in 
both  these  areas.  This  high  failure  rate  also  suggests  that  using  a  combination 
type  contract  should  be  avoided,  since  it  has  a  high  likelihood  of  failure.  Fort 
Eustis  had  the  lowest  failure  rate  of  any  of  the  MICCs  examined  in  this  research 
at  0.42%;  as  seen  in  Chapter  IV,  this  could  be  because  of  a  lack  of  trained 
personnel  to  adequately  enforce  proper  contract  administration.  This  could  also 
be  from  proper  manning  with  adequately  trained  personnel  allowing  for  adequate 
contract  management. 

C.  AREAS  FOR  FURTHER  RESEARCH 

Additional  research  would  be  valuable  in  several  areas  to  further  define 
the  management  levers  that  drive  success  in  service  contracts.  The  database 
used  in  this  research  focused  heavily  on  outcome  successes  defined  by  Hagan, 
Spede,  and  Sutton  (2012).  As  such,  expanding  this  research  to  include  the 
DAWA  training  profile,  numbers  of  assigned  personnel,  and  the  number  of 
CORs  the  MICCs  use  would  add  significant  insight  to  this  research.  This 
research  could  also  be  expanded  to  include  other  DoD  organizations  since  the 
contracts  analyzed  in  this  report  are  only  Army  service  contracts. 

Additional  research  would  also  be  useful  in  analyzing  a  larger  number  of 
contracts  that  received  either  an  unsatisfactory  or  marginal  rating.  Examining  all 
the  contracts  in  the  PPIRS-RC  database  that  receive  a  failure  label  might  yield 
useful  results  when  combined  with  this  study. 
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While  conducting  preparation  for  this  research,  a  large  amount  of 
information  was  found  on  how  government  contracting  has  historically  been 
unsuccessful.  This,  along  with  the  GAO’s  placement  of  contract  management  on 
the  High  Risk  list  since  1992,  causes  the  expectation  of  a  high  service  contract 
failure  rate.  This  data  contained  715  contracts  with  only  22  total  contract  failures. 
This  set  the  total  contract  failure  rate  at  3.08%.  This  suggests  that  additional 
analysis  is  needed  to  determine  if  the  CPARS  database  is  accurately  capturing 
the  performance  of  a  service  contract.  The  results  found  in  this  research  are  all 
based  on  CPARS  entries,  which  could  be  biased  because  of  a  conflict  of  interest. 
This  conflict  of  interest  is  created  since  whatever  rating  the  Assessing  Official 
gives  to  a  contract  may  be  a  reflection  in  the  way  that  person  structured  or 
managed  the  contract.  This  could  potentially  cause  the  rating  a  contractor 
receives  to  be  higher  than  it  should  have  actually  received.  The  findings  in  this 
report,  combined  with  a  service  quality  gap  analysis,  would  add  valuable  insight 
into  what  makes  service  contracts  successful.  The  service  quality  gap  analysis 
would  reveal  the  accuracy  of  the  rating  a  contact  received  in  CPARS,  and  would 
be  useful  in  determining  if  CPARS  accurately  captures  a  contracts  performance. 
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APPENDIX 


These  are  CPAR  Evaluation  Ratings  and  Definitions  from  the  Department 
of  Defense  Contractor  Performance  Assessment  Reporting  System  policy  guide. 


Kalins: 

UcrinitioD 

-Note 

Eiecptlonal 

Performance  meets  contractual 
requuements  and  exceeds  man>  to  the 
Government's  benefit.  The  contractual 
performance  of  tlx  clement  or  sub- 
clement  being  assessed  was  accxrmpILshcd 
with  few  minor  problems  for  which 
corrective  actions  taken  by  the  contractor 
was  highly  effective. 

To  justify  an  Exceptional  rating,  identify  multiple 
significanl  events  and  state  bow  they  w  ere  of  benefit 
to  the  Gov  enmicm.  A  singular  benefit,  however, 
could  be  of  such  magniiuik  that  it  alone  constitutes 
an  Exceptional  raung.  .Also,  there  should  have  been 
NO  significant  weaknesses  identified. 

N  er>  Good 

Performance  mcc’is  contractual 
requirements  and  exceeds  sense  to  the 
Government’s  benefit.  The  contractual 
pcrfotmarrcc  of  the  clcntcni  or  sub- 
clcment  bcitrg  assessed  w  as  accomplished 
with  some  minor  problc.ms  for  which 
corrective  actions  taken  by  the 
coirtrractor  was  effective. 

To  justify  a  Very  Good  rating,  identify'  a  significant 
event  and  state  how  it  was  a  bcitcftt  to  dre 

Government.  There  should  have  been  no  significant 
wcakne.sses  identified. 

Satlsfactors 

Pcrfornsancc  meets  contractual 
requirements.  The  contractual 
performance  of  the  element  or  sub- 
clcnrcnt  contains  some  minor  problems 
for  which  corrective  actions  taken  by  the 
contractor  appear  or  were  satisfactory. 

To  justify  a  Satisfactory  rating,  there  should  have 
been  only  minor  problems,  or  major  problcnia  the 
contractor  recovered  from  w  idiout  impact  to  tlie 
contract  There  should  have  been  NO  significant 
weakne.vscs  identified.  .A  fundamental  principle  of 
assigning  ratings  is  that  contraciors  will  not  be 
assessed  a  rating  lower  than  Satisfactory  solely  for 
not  performing  beyond  the  requirements  of  the 
contract 

Martsiaal 

Pcrfonnancc  docs  not  meet  some 
contractual  requirements.  The  contractual 
performance  of  the  element  or  sub- 
clcnsem  being  asscs.sed  reflects  a  serious 
problem  for  which  ihe  contractor  has  not 
yet  identified  corrective  actions.  The 
contractor's  proposed  actions  appear  only 
marginally  cfTcctivc  or  were  not  fully 
implemcnied. 

To  justify  Marginal  performance,  identify'  a 
significanl  event  in  each  cautgory  that  the  contractor 
had  trouble  overcoming  and  state  bow  it  impacted 
the  Government  A  .Marginal  rating  should  be 
supported  by  referencing  tbc  management  tool  tliai 
notified  the  contractor  of  die  contractual  deficiency 
(c.g.,  management  quality,  safety,  or  enviroomcntal 
deficiency  report  or  letter). 

1  osatJsfactori 

Performance  docs  not  meet  nto« 
contractual  requirements  and  recovery  is 
rut  likely  in  a  timely  marmer.  The 
contractual  performance  of  the  element  or 
sub-clcinem  contaiirs  a  serious  problcnks) 
for  wliich  tlte  contractor's  corrcaivc 
actioiis  appear  or  were  ineffective. 

To  justify  an  Unsalisfaciocy  rating,  identify  multiple 
significant  events  m  each  category  dial  the  eontraetoi 
had  trouble  overcoming  and  state  bow  it  impacted 
the  Government  A  singular  ptubictn.  bowevet, 
could  be  of  such  serious  magnitude  that  it  alone 
constitutes  an  unsatisfactory  rating.  An 

L  nsatisfactory  rating  sltould  be  supported  by 
referencing  the  management  tools  used  to  notify  the 
conuactor  of  the  contractual  deficiencies  (c.g., 
management  quality,  safety,  or  cnvuonracntal 
deficiency  reports,  or  letters'). 

NOTE  1 .  Plu.s  Of  minus  sign*  inj)  be  used  to  indicate  an  improving  (+)  or  worsening  {-)  trend  insuSRcieru  to  cliange  live 
assesstnem  status. 

NOTE  2:  N>  A  (not  applicable)  should  be  used  tf  the  ratings  arc  not  going  to  be  applied  to  a  particular  area  for  evaluation. 
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Rating 


Definition 


■Vote 


Eiccptional 


Ver)  Good 


Satisfactory 


Exceeded  all  negouated  subeonuacdit;! 
goals  or  exceeded  at  least  one  gcxal  and 
met  aU  of  the  other  negouated 
subeontiacting  goab  for  the  eurrent 
ceriod.  Had  exceptional  success  with 
inttiatives  to  assist,  promote,  and  utiluc 
small  busLness  fSB),  small  disadvantaged 
busutess  (SDB).  woinen-owucd  small 
business  (WOSB).  HUBZone  small 
business.  vctctat>H>wncd  small  business 
{\'OSB|  and  scev  ice  di.sablcd  veteran 
owned  small  business  (SDVOSB). 
Cocitplicd  with  F.\R  52.219-8,  Utilixaion 
of  Small  Busincs.s  Concerns.  Excoedod 
airy  other  small  business  participauon 
icquircnrcnts  incorporated  in  the  oomiact. 
including  the  use  of  small  businesses  in 
mission  critical  aspects  of  the  progran'. 
Went  above  and  beyond  the  required 
clenrcnts  of  the  subcontracting  plan  and 
other  small  business  requirements  of  the 
contract.  Completed  and  submitted 
Individual  Subcontract  Reports  and'or 
Summary  Subooncract  Rcpor.s  in  an 
accurate  and  timely  manner. _ 

Met  all  of  the  negotiated  subconiractirg 
goals  in  the  traditional  socio-coonomir 
categories  {SB,  SDB  aird  WOSB)  and  raci 
at  least  one  of  the  odici  socio-econom  c 
goals  (HL’BZone.  VOSB.  SDVOSB)  far 
the  current  period  Had  significant 
success  with  initiatives  to  assist,  promote 
and  utilize  SB.  SDB.  WOSB.  HUBZok. 
VOSB.  and  SDVOSB.  Complied  with 
FAR  52.219-8.  Utilization  of  Small 
Business  Conccras.  Met  or  exceeded  my 
other  small  business  participation 
requirements  incorporated  in  the  contract. 
irKludtng  the  use  of  small  businesses  in 
mission  critical  aspects  of  the  prograir. 
Endeavored  to  go  above  and  beyond  Ue 
required  elements  of  the  subcontracting 
plan.  Convicted  and  submiacd 
Individual  Subcontract  Reports  and  or 
Summary  Subcontract  Rcpcitis  in  an 
accurate  and  tiiady  manner. _ 

Dcmoa.suaied  a  good  faith  effort  to  mtet 
all  of  tbc  negotiated  subcontracting  gcals 
in  the  various  socio-economic  catcgorcs 
for  the  current  period  Complied  with 
F.AR  52.219-8.  Utilization  of  Small 
Business  Concerns.  Met  any  other  snail 
business  particioation  reouircnicois 


To  Justify  an  Exceptional  rating,  identify  multiple 
sigrufkant  events  and  state  bow  they  were  a  benefit 
to  small  business  utilization.  A  singular  bcncnc. 
however,  could  be  of  such  magnitude  that  it 
constitutes  an  Exceptional  rating.  Ensure  that  small 
businesses  ate  given  meaningful,  innovative  work 
directly  related  to  tbc  project,  rather  than  pcnpbctal 
work,  such  as  cleaning  offices,  supplies,  landscaping, 
etc.  Also,  there  should  have  been  no  significant 
wcakitcsscs  KJentifled. 


To  justify  a  Very  Good  rating,  identify'  a  significant 
event  and  state  how  they  were  a  benefit  to  small 
business  utilization.  Ensure  that  small  businesses  arc 
given  meaningful,  innovative  work  directly  related  to 
the  project,  rather  tlian  peripheral  work,  such  as 
cleaning  offices,  supplies,  landscaping,  etc.  There 
should  be  no  significant  weaknesses  identified. 


To  justify  a  Satisfactory  raung.  there  should  have 
been  only  minor  problems,  or  major  probictns  the 
contractor  has  addressed  or  taken  corrective  action. 
There  should  have  been  no  signifkant  weaknesses 
identified  A  fundanicmal  principle  of  assigning 
ratings  is  that  contractors  will  not  be  assessed  a 
rating  lower  than  SatUtactorv  solely  for  not 
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Margintl 


L’auiiifactor> 


incSudcd  in  the  centract.  Fulfilled  the 
rcquiicmcnu  of -iie  subcontracting  plan 
included  in  the  coniract.  Completed  and 
submitted  Individual  Subcontract  Reports 
and'or  Sunaninrv  Subcontract  Reports  in 
an  accurate  and  timely  manner. _ 

Dciictcnt  in  meeting  kc>  subcontracting 
plan  elements.  Oeficient  in  complying 
u'ith  FAR  52.219-8.  UtiUration  of  Small 
Business  Conccros,  and  any  other  small 
business  participation  roquireincnts  in  the 
contract.  Did  not  submit  Individual 
Subcontract  Reports  and'or  Summary 
Subconuac*.  Reports  in  an  accurate  or 
timely  manner.  Failed  to  satisfy  one  or 
more  requirements  of  a  corrective  action 
plan  currently  in  place;  however,  does 
show  an  interest  in  bringing  performance 
to  a  satisfactory  level  and  bas 
demonstrated  a  oommitmeru  to  apply  the 
necessary  resources  to  do  so.  Required  a 
correaivc  action  plan. _ 

Noocooipliam  with  FAR  52J119-8  and 
52.^  19-9,  and  any  other  small  business 
participation  requirements  in  the  contract. 
Did  not  submit  Individual  Subcontract 
Reports  aixibr  Summary  Subcontract 
Reports  in  an  accurate  or  timely  manner. 
Showed  litdc  interest  in  bringing 
pcifotmance  to  a  satisfactory  level  or  is 
generally  uncooperative.  Required  a 
cotTcctivc  action  plan. 


performing  beyond  the  tequirements  of  the  contract. 


To  justify  Marginal  performance,  identify  a 
significant  event  that  the  contractor  had  trouble 
overcoming  and  how  it  impacted  small  business 
utilization.  .A  Marginal  rating  should  be  supponed 
by  referencing  the  actions  taken  by  the  government 
■hat  notified  tire  contractor  of  the  contractual 
deficiency. 


To  justify  an  Unsatisfactory  rating,  identify  multiple 
significant  events  that  the  contractor  Irad  trouble 
overcoming  and  state  bow  it  impacted  small  business 
utilization.  .A  singular  problem,  however,  could  be 
of  such  serious  magnitude  that  it  alone  consiitiucs  an 
Unsatisfactory  rating.  An  Unsatis&ctory  rating 
should  be  supported  by  referencing  the  actions  taken 
by  lire  government  to  notify  the  conductor  of  the 
deficiencies.  When  an  Unsatisfactory  rating  is 
Justified,  die  conctacting  officer  must  consider 
whether  the  contractor  made  a  good  faith  effort  to 
comply  with  the  requirements  of  the  subcondacting 
plan  required  by  FAR  S2.219-9  and  follow  the 
procedures  outlined  in  FAR  52.219-16.  Liquidated 
Damaiics-Subcomracting  Plan. 


NOTE  I :  Plus  or  minus  signs  may  be  used  to  indicate  an  imncoving  (+)  or  worsening  (-)  trend  insufficient  to  change  assessment 
status. 

NOTE  2:  For  subcontracting  plans  under  the  DoD  Comprehensive  Small  Business  Subcontracting  Plan  (Test  Program). 

DFARS  252.219-7004  (deviation),  the  racings  entered  in  CPARS  shall  mirror  those  assigned  by  the  Defense  Contract 
Management  Agency  who  is  responsible  for  monitoring  such  plans. 

NOTE  3:  Generally,  zero  percent  is  not  a  goal  unless  the  Contracting  Officer  determined  when  negotiating  the 
subccotracling  plan  that  no  suboontraccing  opportunities  cxi.<it  in  a  particular  socio-economic  category.  In  such  eases,  the 
contractor  shall  be  considered  to  Iia>  c  met  the  goal  fot  any  socio-economic  category  where  iIk  goal  negotiated  in  the  plan 
was  zero. 
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